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ABSTRACT 

Intended for use by training professionals with 
little or no background in the application of automated data 
processing ( ADP) systems, processes, or procurement requirements, 
this reference manual provides guidelines for establishing a computer 
based training (CBT) program within a federal agency of the United 
States government. The manual covers: (1) needs assessment; (2) the 
technical components of a formal feasibility study, as required by 
federal ADP procurement rules; (3) obtaining management approval; (4) 
implementation planning, in order to meet the additional federal ADP 
procurement requirement for a system proposal; (5) prototype system 
development; (6) courseware development; and (7) the evaluation and 
maintenance of CBT systems. A flow chart giving an overview of the 
process of implement ing CBT programs is provided . Appendices present 
a list of 141 references and resources; a 121-item glossary; lists of 
relevant organizations , associations , consort ia, networks , vendors, 
systems , and languages; summaries of current CBT programs in federal 
agencies; a table outlining current CBT activities in federal 
training; and a list of the Federal Interagency Group for 
Computer-Based Training { IGCBT) organizations and contacts . (ESR) 
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1 . INTRODUCTION 



Whv a Starter Kit? 

Computer-based training (CBT) is making far-reaching improvements in 
the way organizations solve their training needs- However, establishing a 
CBT program is a complex task requiring extensive planning, resources, 
organizational adjustments, and top management commitment . It is a task 
involving many new or possibly unfamiliar issues and potential pitfalls to 
unsuspecting trainers* 

This Starter Kit is based on the collective experience of the Federal 
Interagency Group in Computer-Based Training (IGCBT). As much as we wish 
it were otherwise, it is the consensus of the IGCBT that the general pro- 
cess outlined in this Starter Kit cannot be short cut without great risk* 
We simply hope that the Kit will make the Job of initiating or managing a 
CBT program a bit easier. 

While the Starter Kit includes a variety of subject areas which are 
essential to successful implementation of a CBT program in Federal agen- 
cies, we believe it will be a worthwhile reference to the private sector as 
well* The kit is written for training professionals who, although appro- 
priately assigned to manage a CBT program, come to the assignment with 
little or no background in the application of Automated Data Processing 
(AD?) systems, processes, or procurement requirements. 

About the IGCBT 

The IGCBT was formed in 1978 by individuals from the Office of Person- 
nel Management and several Federal agencies then exploring the feasibility 
of CBT (or conducting CBT research). The main objective of the group is to 
exchange information and share resources- In keeping with this objective 
several participants contributed to this Starter Kit to help organizations 
just beginning CBT programs* As much of the material in the Kit is based 
on approved documents from successful programs designed by our members, the 
reader may find some variance in style and approach from chapter to 
chapter. 



The IGCBT meets on the last Tuesday of each month. Meetings are open 
to Federal employees - Persons from non-Federal organizations may also 
attend through prior arrangements with the chair of the IGCBT. Inquiries 
about the IGCBT should be directed to: 

Larry We 3 t burg, Patrice Walker or Sue Middendorf 
Office of Personnel Management 
Personnel Management Training Center/WED 
Instructional Systems and Technology Branch 
(202) 653-7055 

The opinions expressed in this Starter Kit are the opinions of the 
individuals involved and not necessarily the opinions of the organizations 
which employ them* 

Overview 

Computer Based Training (CBT) is an all-inclusive term for training 
that is both delivered directly by computer (Computer Assisted 'Instruction, 
CAI) and training that is managed by computer (Computer Managed Instruc- 
tion, CMI)* The IGCBT recommends implementing CBT programs through the 
systematic process outlined in Figure 1-1* The process begins with a sys- 
tematic and basic assessment/validation of training needs, which must pre- 
cede any proposed major change in a training program ( chapter 2) * Most 
professional trainers will be familiar with this phase of the process* 
Because CBT falls within the scope of Federal ADP procurement rules, a 
formal feasibility study is required* Chapter 3 describes the technical 
components for such a study in accord with standard requirements* Chapter 
4 recommends an approach for winning management approval with the least 
"pain" and controversy* Chapter 5 describes an approach for implementation 
planning which meets the additional Federal ADP procurement requirement for 
a system proposal * Note: some organizations may have only one formal 
management approval step for the feasibility study and system plan* Chap- 
ters 6, 7p and 8 offer advice on how to conduct prototype projects, design 
courseware, and evaluate your CBT system* Chapter 9 ends with the point 
that CBT systems, unlike most conventional ADP systems, should evolve* The 
planning and implementation process should be carried through many cycles 
as the organization adjusts to and gains experience with CBT. 



Figure 1-1- Overview of Starter Kit 
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2. NEEPS ASSESSMENT 



The determination of whether to use new technology for training must 
begin with an analysis of what the training needs are and how well they are 
being fulfilled by the present system. 

In the application of instructional system design * a training needs 
analysis begins ideally ^ with a comparison of the objectives or require- 
ments for either the proposed or existing training with a recent job task 
analysis. To be valid, the training objectives should be derived directly 
from the process of job task analysis. 

The process of job task analysis involves documentation of the complete 
inventory of job functions performed in a particular job specialty^* Each 
of the tasks is then categorized in terms of values that differentiate the 
importance of each function to the performance of the job* such as fre- 
quency of occurrence , critical loess, etc* All tasks are then grouped 
appropriately in terms of knowledges and skills that are required in order 
to perform the functions they represent* Training and job proficiency 
levels are then established based on the degree to which a new recruit or 
trainee should be able to perform in comparison to the performance expected 
of a fully competent performer (journey level)* For further information 
examine the Interservice Procedures for Instructional System Dev elopment 
Hq3sX (ISDM) (U*S* Training and Doctrine Command Pamphlet 350-30) available 
via GPO* or other relevant publications on job task user task analysis* 
(See reference list* Appendix A*) 



ideally is used here because trainers often fino that a iack of time 
and/or resources prohibits them from utilizing this technique* rational and 
logical as it is* 

2when resource constraints prohibit job task analysis* a job function 
analysis (JFA) is sometimes performed* This process contains much less 
detail than a job task analysis, A JFA is based upon job functions, but 
may be assembled by a review of organizational directives* official po- 
sition descriptions* limited interrogation of field experts* and other 
locally available expertise* 
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Effectiveness » Efficiency and Cost 

Assuming that current training objectives have been derived from & 
valid job taak analyaia, the next queation to aak ia whether conventional 
classroom or other training delivery methods are currently effective, or 
atated another way, are employeea performing aatiafactorily? If the answer 
ia yea, or if there ia no information available, then the coat and/or 
ef fectiveneaa of varioua training alternatives ahould be examined* 

However, an examination of the coat/ef fectiveneaa of CBT (or some other 

new technology) could alao be motivated by a deal re to maxirize training 

effectiveneaa to reduce or minimize overall training coata and/or the time 

required to attain varioua levels of proficiency * A common reaaon for 

using computer* baaed inatruction ia an expected redaction of atudent and 

inatructor time and the reaulting coat aavinga* HoweveJ), other equally or 

more important benefita may include training effectiveness (without which a 

coat reduction ia meaningleaa) ; atandardization of course delivery; aimu- 

lation of computer related job environment; modularization and individual ~ 

pacing to accomodate varied entry levels, Job reaponaibilitiea and learning 

akilla; automated teat grading and record keeping; eaae in delivery of 

V 

courae changea and adaptatlona; and remote delivery capability in field or 
on-the-job locationa* 

If your current training ayatem ia operating effectively, you will need 
to make a detailed analyaia of that ayatem to make a determination of 
whether it would be coat beneficial to move to CBT or other new technology 
aa a delivery or management medium* To help you conduct that analyaia we 
have provided a liat of training ay 3 tern functional requirement a later in 
thia chapter* However, if employeea in the organization are not performing 
current job taaka aatiafactorily, you muat f irat conduct a performance 
analyaia* : 

Performance Analysis 

Thia proceaa cauaea the training manager to explore whether the pro-* 
posed training need ia valid* That ia, ia there a need for the employee to 
increaae his or her level of knowledge or akill in the job taak or ia 
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the performance deficiency related more to environnfentar\conditions r lack 
of motivation or other organizatinal factors which inhibit adequate per- 
formance f such as the way a job task, or function or procedure is con- 
structed or managed? For additional information on conducting a perfor- 
mance analysis, consult the relevant publications by Robert Mager and J* H* 
Harless (See Appendix A)* 

^ If the performance analysis helps to dooument a valid training need 
that "Us not being met by the current training process, then an evaluation 
of alternative training delivery and/or training management techniques is 
in order* (The reader may want to review the inter-service ISD model (op* 
cit*) for information on selecting instructional delivery systems, or look 
at the training system function requirements in the following section) * 
Once these preliminary steps have been taken and CBT is considered a valid 
option for training delivery and/or training manageuent* the reader is 
advised to follow the recommendations included in Chapter 3» Feasibility 
Study* 

Training System Functional R equirements 

The following list /checklist of important aspects of an instructional 
systems design (ISD) -based training system may be assistance in diagnosing 
whether your current or proposed system of training is effectively addres- 
sing your agency's performance requirements* This type of analysis should 
also help you later when you have to functionally describe CBT system 
requirements* 

Following job task analysis and training needs analysis* ISD-based 
training systems can be divided into seven functional areas and the re- 
quirements established for each* They are: 

—Course development 

—Lesson delivery 

— Testing 

—Recordkeeping 

— Managing student progress 

— Course maintenance and revision 

— System and program interfaces 
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Course Development . The training ayatem muat provide for courae 



development which: 

1 . Followa the inatructional ayatema deaign model (i.e. , 
training plana and leaaona are designed to meet the 
apecific requirementa of a job atandard or analyaia 
contained in a training proposal)* 

2. Supporta modularization of couraea. 

3- Allowa aimultaneoua leaaon delivery and development 
activitiea. 

4 . Allowa for dynamic growth, improvementa , and maximum 
utilization of available methodology/technology, e.g., 
simulators. 

5. Allowa for the uae of already developed outaide courae 
material . 

Leaaon Delivery . The training ayatem muat provide for leaaon delivery 

which: 

1 . Enaurea standardization. 

2. Haa the ability to teach cognitive and procedural skilla 
through the following kinda of learning: apecific re- 
sponding, motor chaining, verbal chaining, discrimi- 
nating, classifying, rule uaing, and problem aolving. 

3. Specif iea procedurea for conveying information that 
impacts on the more emotional or affective dimenaion of 
learning. Thia learning might include human relatione, 
auperviaory/ employee transactions, optimal (satis- 
factory) productive, motivation, EEO and/or LMR* 

4. Allowa the atudent to move through the courae material 
aa quickly as poaaible and at the highest level of 
quality poaaible. Thia ia accompliahed uaing a variety 
of techniquea. 

— Modularization 

— Individualized inatruction 

— Immediate feedback 

—Interactive leaaona 

— Uae of multi-media techniquea 

— Efficient atudent counaeling 

—Uae of motivating leaaon material and training media. 
— Uae of high tranafer slmulationa and games. 

5* Allowa for the identification and analyaia of couraeware 
probl ema. 

6. Allowa aimultaneoua delivery of multiple leaaona. 
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7* Freea inatructora from repetitioua classroom teaching or 
course management activitiea. 

8. la readily accessible to students (and courae graduates 
for refreaher training). 

lasting. The training ayatem muat provide for teating which: 

1. Meaaurea acudent maatery of inatructional objectivea 
through queationa or akilla demonatrationa. 

2. Meaaurea atudent progreaa in a valid and reliable 
manner. 

3. la acored quickly to provide feedback and timely leaaon 
assignments. 

U, la aecure from compromiae. 

5. Preventa - the atudent from "teat queation learning" 
through random generation of a variety of questions, 
problems or skill performance aituationa. ( "Teat quea- 
tion learning* may occur if students are repeatedly 
expoaed to the same teating materials.) 

6. Provides atatiatical analyaea neceaaary to determine 
queation validity and reliability. 

Recordkeeping and Analy si s . The training ayatem muat provide a record- 
keeping function which collecta the data and performa the atatiatical analyaea 
neceaaary to: 

1. Determine individual atudent and group progreaa, 
accomplishment and grade. 

2. Schedule atudenta and manage training reaourcea. 

3* Determine validity and reliability of teat queationa. 

4. Analyze inatructional effectiveneaa. 

5. Determine inatructional coata. 

6. Develop, maintain and reviae courseware. 

7 . Develop courae or program evaluation and^ activity 
reporta. 

Managing Student Progress . The training ayatem muat provide for manage- 
ment of atudent progreaa by: 

1. Identifying entry level. 

2. Generating individual leaaon &3aignmenta. 
3* Identifying need for atudent counaeling. 

. Predicting and analyzing atudent progreaa. 
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5. Providing warning when a student is not progressing as 
scheduled. 

6. Monitoring many students simultaneously. 

7. Providing administrative structures which motivate 
students to complete course in a timely manner. 

8. Allowing an instructor to handle a large number of 
students. 



Course Maintenance and Revision. The training system must provide for 
course adjustment, maintenance and revision through; 

1. Complete and self-explanatory documentation of course 
material development to ensure continuity. 

2. Systematic updating of all distributed course material. 
3* Course modularization. 

4. Analysis and reporting mechanisms to identify course 
deficiencies precisely. 

5. Easy updating process. 



System and Program Interfaces. The training system must provide for 
interfaces with related administrative and operational programs and provide a 
means of handling or adopting the considerations of the learning environment 
by: 

1. Providing flexibility to handle fluctuating number of 
students. 

2. Supporting instructor and student learning and career 
growth. 

3. Providing efficient transfer of data to other 
information systems. 

4. Minimizing the amount of time employees are absent from 
the job site due to training. 

5 . Operating with common~use , continuously updated and 
improved systems so that the agency may readily contract 
for course development and exchange courseware with 
other Government or private training institutions. 
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1. FEASIBILITY STUDY 



Once your have decided that a CBT ayatem might meet your organization^ 
training needa you will need to aecure the neceaaary procurement authority to 
obtain a computer ayatem for which a feasibility atudy will be required. Thia 
ia normally reviewed by all levela of an organization, and then by General 
Servicea Administration (GSA) . The extenaiveneaa of the review generally 
dependa on the dollar amount of the procurement. 

Thia aection ex plain a what ia required in a feasibility atudy , and de- 
acribea thrte other studies which are usually attached to or related to the 
feaaibility atudy. They are the telecommunication atudy, the A-76 analysia 
(baaed on the Office of Management and Budget'a Circular A-76), and a ayatema 
atudy . Although theae are required government procedural documenta, element a 
of the&e atudxea probably will be required of anyone who ia attempting to 
acquire a CBT ayatem for hia or her organization. Note that all of the atudy 
requirementa aak the propoaer to deacribe a ayatem in ADP terma. Thia ia a 
problem for trainera who are not accustomed to deacribing their incorrectly 
reaponding atudenta as "outputa". Care muat be taken to avoid confuaion be** 
tween an "inatructional" ayatem and a "computer" ayatem. 

Study Components 

Each computer ayatema feaaibility atudy ahould include the following 
information: 

Hili- Include the full name of the computer ayatem and acronym if one 
ia uaed, e.g., Computer-Baaed Training Syatem (CBT). 

Identification . Indicate originating office, including geographical 
location, parent organization, and preparation date. 

Problem Statement and Relation to Mission. Define the training problem 
or need in apecific, clear terma. Uae quantitative data if poaaible. If the 
requirement ia for additional reaourcea for an on-going ayatem, indicate accom- 
pliahmenta to date and the need for continuing the project. 

Provide rationale to explain why the problem ia one which the organi- 
zation muat try to aolve. Explain how the accompliahment of the organization 
miaaion haa been adveraely affected by the problem. 



Problem Analysis. 



1. Description. Describe why the problem is occuring and 
the consequences. For example, an increase in workload 
may not be a problem if the existing staff or systems 
can handle it within the necessary performance stan- 
dards. The basis for workload forecasts should be de- 
fined in terms of how they relate to staffing and budget 
requests. For on- going computer systems and equipment 
upgrades, provide a summary report which highlights the 
critical points and the rationale for change. The re- 
port should be based on a minimum of three months of 
recent quantitative performance evaluation data. 

2. Magnitude. Describe who the problem affects and how 
wide-spread the problem is. Identify as specifically as 
possible the population affected. 

3* Casual Factors. Discuss the factors causing the prob- 
lem. Are they beyond the organization* 3 control? Are 
they likely to persist or are they temporary? 

4. Data Sources. Identify sources of the data on which the 
conclusions stated in the Feasibility Study are based. 
Where necessary* full documentation of the data sources 
may be required. 

Detailed Analysis of Proposed Program or Detailed Requirements Analysis 

1 . Objectives. State the objectives of the proposed pro- 
gram in measurable, quantitative terms , e. g* , implement 
CBT (proposed system) in three cities, or train so many 
employees in a specified time frame. 

2. Expected Accomplishments. If the objectives are 
achieved, to what extent will the problem be resolved, 
e.g., increase student mastery levels by 10 percent. 

3. Assumptions. Discuss the assumptions on which the pro* 
posed program is based. 

4. Description. Describe how the proposed program will 
work and how it will alleviate the causes of the prob- 
lem. Discuss those things that will be done to imple- 
ment the program, who will implement it, and how the 
steps will be accomplished. Be specific and provide 
details. A high-level flow chart and a narrative des- 
cription of the major steps and their relationship in 
the proposed program should be provided. Indicate the 
flow of information and the relationship with users and 
other systems. Describe each major input and output in 
terms of purpose, format, and source. Response time 
requirements for classes of outputs should be prov ided* 



Indicate proposed file structures and any unusual pro- 
cessing requirements. 

5. Implementation Plan. The implementation plan should be 
provided and end products should be identified. A 
simple milestone bar chart or equivalent should be 
provided. 

6. Impact and Performance Measures . Performance measures 
are defined as those which show whether the proposed 
program is bei ng carrie d out as planned . For example , 
is CBT (proposed system) fully implemented on schedule? 
Impact measures are defined as those which show whether 
the proposed program is having the desired effect. For 
example f did student mastery levels improve 10$ in the 
(proposed) CBT system? Is it alleviating the problem? 
Describe both types of measures for the proposed pro- 
gram. 

7 • Resources. Provide an overview of the resources ( per- 

sonnel, money, equipment, etc.) needed to implement and 
operate the proposed program. 

8 - Constraints, Risks and Uncertainties. Identify the 

factors which may impede the success of the proposed 
program . Factors beyond the control of your organi- 
zation would be discussed here. 

9. Systems Life. State the expected systems life time of 
the proposed program. A minimum of six years should be 
provided. 

Analvaia of AI ternatlvea. 

1 . Identify Alternatives. List and describe alternatives 
for alleviating the problem. Generally, if there 
appears to be only one way to solve the problem f the 
problem has not been adequately defined. See, for 
example, Figure 3-1 displaying pros and cons of various 
alternatives for instructional delivery taken from the 
feasibility study of The Federal Aviation Administation. 
The alternatives must consider facilities available 
under the Federal ADP Sharing Program and the use of 
commercially available services as required by 0MB 
Circular A-76- 

2. Analyze Alternatives. Estimate the cost for each alter- 
native, time to implement and the estimated impact on 
the problem. Discuss the advantages and disadvantages 
of each alternative and explain why the proposed program 
was selected. 
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Figure 3-1, ANALYSIS OF ALTERNATIVES 



Evaluating Opt ions - The Pros 



STATUS 

quo 



MANUAL 
TNDLV . 



Avoids Consequences of Change 

Cut Student's Time and Related Costs 

Builds on StudenL 1 s Background/ 
Training 

Easy Course Maintenance fii Security 

Standardized Ins true t ion 

Cuts Lnstructor Time & Related Costs 

High Quality Interactive Instruction 

Capabi 1 i ty for Hand ling Additional 
Courses 

Students Operationally Av ail able 

Easy to Handlo Peak Student Loads 

Efficient Eva 1 uat ion by Field 

Maximum Travel/Per Diem Savings 

May Support Proficiency/Field 
Generated Training 



CENTRAL 
LOCATION 
CM I ONLY 



X 
X 
X 
X 



CENTRALIZED 
LOCATION 
CAI/CMI 



RE MOT ED 
CAL/CMI 



X 
X 
X 
X 
X 

X 
X 



X 
X 
X 
X 
X 

X 
X 
X 
X 
X 



Evaluating Options - The Cons 

No Room for Additional Courses X 

FY 1980 Constraints Causing 

Cancel lat ions X 

Student Scheduling More Complex XXX X 

Providing for Student Interact ion 

Difficult XXX X 

Course Deve lopment More Demand ing X X X X 

Subs tant ia 1 Cover s ion Costs X X X X 

Large Investment in ADP Equipment/ 

Training XX X 

Need for CAl/CMI Authors X X 

Management of Multiple Learning 

Centers X 

Beck, John A • and Beardsley* Patricia S M "Hov Computer Based Training was Cost-Justified in 
the Federal Aviation Administration/ 1 ADCIS, Proceedings, 1 980 , pp. 98*14. 
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Detailed Resource Requirements - Indicate the equipment, software, per- 
sonnel, and other costs required to achieve the proposed objectives- Costs are 
to be provided for the current system (if any), the development period and the 
proposed system operation- Costs should be calculated in the same fashion as 
if an 0MB required-A-76 study was being prepared for each year of the proposed 
systems life* 

1. Present Systems Costa (if applicable)* 

(a) Personnel 

(b) Equipment 

(c) Site 

( d) Communications 

(e) Other, including supplies 

2. Development Costs. 

(a) Personnel 

( b) Equipment 

(c) Site 

(d ) Communi cations 

(e) Other, including supplies 

3. Operations Costs. 

(a) Personnel 

ib) Equipment 

(c) Site 

(d) Communications 

(e) Other, including supplies 

Milestones . The implementation plan should be subdivided into specific 
or logical segments. Target dates should be assigned. A milestone bar chart 
is usually adequate- For very complex systems, a PERT* type schedule showing 
tasks , duration of each task , start! ng time for each task relative to the 
beginning of the project, interrelationships between items and sub-items, etc-, 
would be more appropriate. 



External Impact « Implications for other divisions, organizations and/or 
agencies should be included here* 



Deliverables/End Products . Deliverables and/or end products should be 
identified* Reports, software packages, and equipment are examples of deliver* 
ables and end products. Indicate whether the Government or a vendor will own 
the deliverables when the work is completed. 
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Telecommunications Study 

If the proposed system is to include telecommunications, the cost of 
required telecommunications support must be estimated. This may be included as 
part of the Feasibility Study or accomplished in a separate, but related, 
document. In order to estimate these costs* telecommunications workload 
measures should be specified and projected over the system life* Alternative 
telecommunication configurations should be sized to support estimated workload 
and the least cost alternative should be identified. Agency telecommunications 
coordinators are a good source of expertise for this type of study* Telephone 
comPany systems engineers may also be used when in* house resources are una- 
v ail able. 

Qu antify and Geographically Distribute the Workload , As a first step, 
the communications traffic must be quantified by geographical location* In 
order to achieve this distribution* a list of terminal locations is necessary 
and the types and quantities of each terminal and supporting communications at 
these locations should be specified* You may encounter many new terms which 
you will have to ask your communications consultant to explain* For example, 
listed below are some of the types of terminal or communication functions that 
should be considered: 

( 1 ) Remote Batch 

(2) Interactive/ Conversational 

(3) Transaction Processing 
( J| ) Graphi cs 

(5) Message Switching 

(6) Bulk Data Transfer 

(7) Line Concentration 

Estimate Workload bv Terminal Location. If possible* within the con- 
straints imposed by the quality of the data, the base year workload should be 
estimated by terminal location, quantified in terms of the number of trans- 
actions originated at each site by month* Once monthly transactions volume is 
known, character transmission volume can be specified by site* 

Additionally » terminal occupancy time should be specified for terminals 
at each site* This data, combined with character transmission volumes* will 
enable the analyst to calculate line occupancy times and to configure communi- 
cations networks accordingly* 




Workload projections should be completed for each existing and proposed 
terminal location* (However, if the expected number of such sites is large, or 
if a significant number of terminals are clustered in a given area, it may be 
necessary to aggregate site workloads for volume projection on a basis which 
appears reasonable to the user) - Upon completion of these analyses, both 
terminal and line occupancy projections will have been identified. These fore- 
casts will then permit generation of al ternative network configurations for 
detailed cost analysis* 

Network Alternatives * Examples of the types of service used in alter- 
native network configurations include: 

( 1 ) Dedicated (leased) line service from common carriers 
(either analog or digital) 

(2) Specialised carrier service such as Value Added Net- 
works (VAN) 

(3) Dial/FTS/WATS network service 

The first of these alternatives represents a network limited to the users 
in the procuring agency * Multi-drop lines and concentrators could be con- 
sidered for analysis* In the second alternative, the trunking facilities only 
are supplanted by specialized carrier services (which are not currently com- 
mercially available at all locations)* The third alternative examines the use 
of dial facilities; the choices are ordinary dial, the Federal Telecommuni- 
cations System (FTS), the Switched Digital Service or the Wide Area Telephone 
Service (WATS) * This list of alternatives is not exhaustive; others which 
appear feasible to the analysis team should be defined and costed* 

Cost Elem ents of the Analvala. Cost elements which would necessarily be 
included in the analysis include type of service, modems, line terminations, 
concentrators, mileage (for dedicated circuits), frequency and duration of use 
(for dial-up circuits)* These cci>tr; should be aggregated for each year of the 
system's life cycle* The least cost equipment configuration identified should 
normally be selected* Additionally, a telecommunications cost must be 
developed for the commercial processing alternative considered as part of the 
OMB Circular A-76 analysis, requiring design of a local line configuration to 
connect ADP terminal sites to the proposed commercial network* 
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The A-76 Analysis 

If the provisions of OMB Circular A-76 are applicable, then an A-76 study 
must be included* In the A-76 study the cost /benefit characteristics of pro- 
cessing on a commercial service bureau will be compared to those derived for 
the lowest 1 - cost internal {Government provided) processing alternative- The 
A-76 analysis must develop a projection of total computer expenditures under 
both the internal and the commercial processing scenarios* These expenditure 
forecasts may have to reflect the expected impact of inflation for all cost 
elements where prices are not fixed contractually - 

The alternative with the lowest present value total oosO should normally 
be recommended as the most cost-effective course of action available to the 
Government- The total cost of the internal processing alternative remains the 
same as that developed in the economic analysis of the Feasibility Study- Con- 
sequently, the purpose of the A-76 is really to estimate the commercial service 
bureau cost of processing the same workload volumes at the same level of per- 
formance described in the workload forecast section of the Feasibility Study- 
The cost elements considered are identical to those listed in Feasibility 
Study* 

Assumptions When Coating External Service . Costing external service 

becomes the principal problem encountered in performing the A-76 analysis* In 

order to accomplish this step of the analysis at least two critical assumptions 

are generally required: 

1* Where no empirical data exists to suggest a better al- 
ternative assumption, the analysis is built upon the 
premise that user demand for computing resources is not 
altered by variations in the billing algorithm (i-e- , 
changes in the relative prices of resources) - 

2* The £ sumption is made explicitly that any processing 
arrangements which are available to the user at a 
Government operated computer center will also be made 
available to the commercial sector- In other words, it 
is assumed that a job requiring twelve 9 track tape 
drives will be accorded identical treatment in both the 
internal and commercial environments- Additionally, no 
premium rates or scheduling problems would be expected 
for computing tasks requiring "unusually high" simul- 
taneous device allocations* 



^The procedures in your organization may or may not require that 
you consider the impact of inflation and/or present value- 



Both of these related assumptions involve significant abstractions from 
reality* However, any alternative treatment of these issues would be equally 
arbitrary * 



Deriving Processing Costs . Once the assumptions stated in the previous 
sub-paragraph are made, the processing costs can be estimated by applying the 
following methodology: 

1* First, the workload estimate derived in the Feasibility 
or System Study must be converted into the units used by 
one or more "representative* commercial service 
bureau(s) * 

2* Having quantified the workload in terms compatible with 
a representative commercial service billing unit, the 
workload estimates may be multiplied by the service 
bureau rates to obtain the cost of processing exter- 
nally* Assuming the representative service bureau is a 
participant in the GSA Time Sharing Services, GSA should 
be consulted to determine the currently prevailing dis- 
count rate to be applied to these prices* 

3 * Include shut down costs of any existent systems which 
would be replaced by the commercial service bureau* 

Primary Processing Costa . In order to estimate the processing costs, at 
least four major cost categories should be considered: 

(1) CPU time charges 

(2) Terminal connect time charges 

(3) Mass storage charges 

(a) shared packs 

(b) private packs 

(10 Magnetic tape utilization costs 

Deferred and priority processing requirements may be estimated if sup- 
porting data can be secured at a reasonable cost* Generally, the assumption 
can be made that deferred discounts and priority premiums will cancel each 
other out over the system life cycle* 



Rft^flPIBft nda fc i Qn - The least cost alternative should be recommended as the 
most cost effective to the Government, given the implicit assumption underlying 
the analysis that both alternatives will be equally beneficial* 
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Svatema Study 



For large procurements, after the Feasibility Study haa been approved, a 
detailed Systems Study should be prepared which will refine, and in some caaea 
redevelop, the groaa eatimatea provided in the Feaaibility Study. The reaulta 
of thia atudy ahould preaent management with a conciae aummary of the following 
elementa: 

— The proposed system* Recommendations for additional reaourcea 
ahould clearly preaent the beat eatimate of equipment soft- 
ware, telecommunication! maintenance , or other reaourcea 
needed to fulfill the ADP need* 

— Input/Output* Data which will be input to the ayatem ahould 
be clearly identified aa to purpose, magnitude, form and geo- 
graphical location* Outputa ahould be aimilarly aummarized* 

-'-Benefits* Itemized improvementa or v aavinga from the propoaed 
acquisition ahould be atated in measurable tenoa* 

— Coats* All expenaea reaulting from the derelopment, inatal- 
lation, and continued operation of the propoaed ayatem ahould 
be itemized* 

— Alternatives* All alternatives that have been examined ahould 
be diacuaaed in tenna of why each waa leaa aatiafactory than 
the propoaed ayatem* 

— Operational Impact. The impact of the propoaed ayatem on the 
preaent system, including peraonnel , procedurea, software, 
current equipment, and privacy, ahould be preaented* 

The information enables management to make an informed decision to recom- 
mend the acquiaition of all, part, or none of the computer reaourcea, that 
would be neceasary to moat effectually implement the computer-' baaed training 
requirements identified. Studiea of data proceaaing needa ahould provide 
managera an opportunity to conaider new and varied methoda of data proceaaing 
and to identify a data proceaaing approach beat suited to the organization's 
training needa. Training requirementa ahould be apecified in a way that 
clearly differentiatea them from conventional data processing. For example, 
the human factora requirementa of the ayatem need to be defined so that it will 
not be aaaumed that "any" terminal ia an effective learning device for a non- 
computer apecialiat or that training ataff can uae any atandard programming 
language to implement computer-baaed training programs. 
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U. GETTING APPROVAL 



Genera l 

Establishing a computer-based training system can be an exciting process 
but it can also be frustrating, intricate, and maddening. Computers are new to 
the traiiiing environment and are often consciously or unconsciously perceived 
as threatening to the positions of others or as dehumanizing mechanical mon- 
sters that defy all understanding; monsters are incapable of improving the 
learning process. 

If they are to avoid substantial organizational resistance t CBT program 
managers and developers must strive to explain their projects very simply. 
Complex explanations can breed confused questions in unproductive cycles. 

Simplicity 

Based on the personal and unscientific (but t we think* accurate) obser- 
vations of several IGCBT members , we have concluded that persons involved in 
establishing new CBT systems frequently have a hard time succinctly explaining 
what they're about. Because they often unwittingly find themselves in an ad- 
vocacy or change-agent role, CBT practioners, when given the choice, sometimes 
prefer to give a long-winded rather than a short-winded answer to the 
questions; What is C5T? (The authors freely admit that they, too, share this 
proclivity.) Unfortunately, this tendency can cause major trouble as the 
practi oners try to overcome the intrinsic organizational resistance to CBT. 
This resistance arises from the fact that CBT is a significant departure from 
conventional training techniques. 

Hanging K.X.S.S. (Keep It Simple, Stupid) signs around the office doesn't 
help. In this chapter we hope to suggest some more effective solutions to the 
"explaining 11 problem. 

What' « the Requirement? 

No amount of extolling the virtues of CBT really clarifies anything if the 
basic training requirement is not understood. Without a training requirement, 
there is no need for CBT. The audience at an organization's Internal presen- 
tations on CBT may not be very knowledgeable about its training requirements. 
Focusing on the actual training requirements of your organization is one of the 



keys to keeping listeners' minds off of the relatively unimportant details 01 
how CBT operates* CBT cannot be explained simply aad clearly if it's the only 
concept receiving attention* 

This approach of talking about CBT by presenting it within its greater 
training context may seem obvious* but we f ve found few CBT practiorers who do 
it . Presenters may wrongly assume their audience understands the training 
context* they may not understand the importance of presenting a CBT program as 
a response to a definitive training need* 

Overall Logic 

The following format is one we recommend for internal presentations on CBT 
to decision makers* Formal presentations should be short* e.g. , no more than 
10-15 charts. Keep back-up charts in reserve to deal with questions but don't 
volunteer them* 



1 . Requirement 

What is the overall mission which training must support? (The 
answer should be obvious to everyone at the presentation which 
makes it a good place to start*) What knowledges and skills must 
the workforce learn? What number of students are expected in the 
next few years due to retirements* turnover* hiring for new pro- 
grams, etc*? Would it be beneficial to reduce the current travel 
or instructor reqirements? How well is the current training sys- 
tem doing or expected to do in supporting the overall mission? Is 
there already a problem in training due to a shortage of re- 
sources? Don't start with an explanation of how CBT "works" or 
how any other training works* 

2* Potential Adverse Impact 

What will happen if yoa don' t do something about the problem? 
This question must be answered in terms of the organization's 
mission It may be difficult or impossible to prove a direct link 
between training and workforce performance because of other fac- 
tors such as the typical unavailability of reliable evaluation 
data related to conventional training* In that case a concensus 
subjective judgement may be offered*^ 



5see Johns Manvills* This excellent empirical study may be usei ul as 
back-up information* It shows that structured training is less expensive than 
unstructured (or "buddy system") training and that students trained in a struc- 
tured manner solve on-the-job problems much more efficiently* 
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3. Wava to MeAt the Reouirement 

Discuss the various ways of delivering the required training, 
including CBT. Show in overview fashion how the use of CBT would 
change the current training system* Don't neglect to mention the 
ways in which computer management of training can provide instant 
evaluation information for managers. Only then define CBT in a 
very summary fashion, fijJUi show a picture. Save any further 
explanations for demonstrations (see below) • CBT is now used by 
so many organizations that for reasonbly informed managers its 
validity as an effective training medium should not be in ques- 
tion. Because of its potential to distract from the objective of 
the presentation, don't offer to discuss validity in a group pre- 
sentation, unless someone really wants to know. In any case, a 
pilot project with the purpose of prov ing that CBT "works* is 
clearly not justified (unless the application is highly unusual)* 

4. Coat Effectiveness 

This is a standard requirement for a staff presentation. In one 
chart show the estimated costs of alternative ways to meet the 
training requirements. Your alternatives will depend largely on 
the ways your organization currently delivers training, but you 
may wish to introduce the cost/ benefit of other technologies in 
addition to or apart from CBT. Your analysis might alco include 
the cost benefit of centralized versus decentralized training. 
Another chart should summarize the many nonquantif iable pros and 
cons of CBT. In some cases these considerations may be a more 
important consideration than cost savings. (Figure 3-1 is an 
example of this kind of chart.) 

5. Implementation 

Show overall timeframe. The details of how the developers will do 
their work should be kept as back-up information. Show summary 
budget data. 

6. Recommendation 

Summarize the de"clsion(s) you want. (Remember to amile.) 



Feeling for CBT 

So much for logical "left brain* concerns. Don't forget your audience f s 
intuitive or "right brain* side. It is necessary to provide demonstrations of 
CBT to key decision makers. In each demonstration the decision maker should 
have time to sit one-on-one with a terminal and view courseware at least some- 
what related to his or her organization* gaining some hands-on experience that 
is similar to that of a "typical* student. Carefully planned, this activity 
does more to overcome conscious or unconscious computer resistance/fears than 
anything else. 
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If many decision makers (we're referring to key staff as well as manage- 
ment) are involved, as they generally are with CBT, this demonstration process 
can take a lot of time. However, it is definitely worthwhile* We doubt that a 
controversial CBT proposal can be accepted in any organization without a 
thorough demonstration process* 

Each demonstration should be analyzed and constructed as thoughtfully as 
your best training course except thac the "training outcomes' 1 should be stated 
in "right brain" terms. What experience, impression, pictures, feelings, etc., 
do you want transmitted? fir ffi t does each person receiving the demonstration 
know what it feela like to take a stimulating CAI lesson? (They don't neces- 
sarily have to take the whole lesson to get that feel.) 

Condition 

Starting or growing a CBT program is a complex undertaking involving many 
organizational adjustments* It should be remembered that resistance to new, 
complex technology is frequently encountered* Analyze your strategies for 
introducing the CBT technology carefully. Know your audience* What "works" in 
one situation may not work in another* Some people respond more to hands-on 
experience while others require additional discussions or presentation* In any 
case, the difficulties of the undertaking may be reduced if the staff respon- 
sible for the program continually strive to keep their explanations of what 
they are doing as clear and simple as possible by emphasizing the greater con- 
text in which they are working. 
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i. SYSTEM PLANS/ PROPOSALS 



The feasibility study described in Chapter 3 will probably require an 
overview implementation plan to be developed for each serious alternative; 
however, once the feasibility study is formally approved f much more in-depth 
implementation planning will be required. Automated data processing ( ADP) 
procurement rules require studies of a number of considerations and issues. 
Some organizations may require these studies to be wrapped up in a single doc- 
ument known as a "system proposal 1 * before they begin the procurement process. 
Others may link the studies more directly with preparation and approval of a 
request for proposal (RFP). 

The IGCBT recommends starting out with "small* prototype projects (see 
Chapter 6) and building the CBT system through a series of discrete evo- 
lutionary steps. Unfortunately this approach and other aspects of good CBT 
system development may be at odds with the approaches used standardly by the 
segments of the organization which normally control the procurement of ADP 
equipment. Many organizations with CBT programs (both Government and private) 
have experienced (or continue to experience) difficult internal conflicts 
between "CBT'ers* and "ADP* era*. 

The best way around the conflict seems to be to do "what everybody wants." 
In other words, make sure you get approval in the feasibility study to launch 
at once into limited prototype projects (i.e., what the trainers want) at the 
same time the overall system proposal (or plan) is being studied and prepared 
(i.e., what the computer specialists want). Lease as much equipment as pos- 
sible for the prototype projects to avoid a premature commitment to ADP equip- 
ment. Under this approach, you can get the program moving through the proto* 
type project and start the arduous process of breaking down organizational 
resistance ( see the importance of demonstrations discussed in Chapter 4) . If 
the prototype effort and the overall system planning process occur simul- 
taneously, the two efforts can complement each other. 

This chapter offers suggestions for designing an overall system proposal 
which responds to the various requirement of the ADP procurement process: 

, _J 

System Requirement V y 

After summarizing the background of your project, your system proposal (or 
RFP package) will need to describe the system you desire in terms of input/ 
output and hardware/software. 



Don't be thrown by those terms. Basically a CBT system is a hardware/ 
software package which Is capable of supporting the development and delivery of 
training courses or course components. some IGCBT members make a distinction 
between software and courseware and between computer programmers and CBT 
authors. Specify the functional , not the technical , characteristics of the 
authoring system you want your authors to use. The authoring system should be 
of f*the-shelf ; it's very expensive to invent your own or even modify an exis- 
ting one. 

Input/output as it applies to CBT must be defined in terms of courses and 
students; courses being the input and trained students being the output. This 
kind of technical computing terminology and logic is so strained and artificial 
as to be of little value. Avoid it by describing what you want the system to 
do in functional terms (i.e., good old "trainingese* 1 ) . Your original training 
requirement analysis should be a considerable help. 

System Flow 

A conventional computer system proposal usually contains elaborate flow 
diagrams detailing the flow of all data elements in the system. That f s not a 
practical approach for CBT applications, particularly if you're specifying an 
off-the-shelf authoring system. However, we suggest you chart the intended CBT 
course development and administration process. The results of that exercise 
will be useful in preparing the new policies, procedures, and orientation 
packages you will need for your CBT system. 

System Impact 

You will need to identify any other computing or telecommunication systems 
in your organization that the CBT system will interact with, ari that's really 
all a conventional proposal is interested in. The major impact of a CBT 
system, however, is on the traditions, limits, career ladders, administration, 
etc., of your overall organizaion. It is advisable to prepare for the likely 
organizational changes by summarizing them up front. 

Privacy Act 

You will need to consult with your organization's Privacy Act coordinator 
to see how your organization interprets the Privacy Act. The simplest position 
is the blanket statement that all records maintained in the CBT system are 
covered under the 0PM/Government*1 General Personnel Records System. 
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Reoiiftah for Proposal (RFP) 

Whether you launch into preparing RFP's as soon as your feasibility study 
is approved or go through the intermediate step of preparing a system proposal, 
you will eventually have to define your CBT system requirements to vendors 
through some procurement process. You can in fact use less time consuming 
procurement processes than RFP's — such as a two~step invitation for bid (IFB) 
if you are Quite sure about your exact requirements. In either case it will 
involve language and concepts from the computer world. Figure 5-1 shows the 
format of a typical RFP. 



Figure 5-1, Typical RFP Format 

RFP TABLE OF CONTENTS 

A. Cover Sheet, standard Form 33 

B. Representations, Certifications) and Acknowledgements 

C. 1. Solicitation Instructions and Conditions 
2. Format Instructions 

D. Glossary 

E. special Provisions 

F. Mandatory Requirements 

G. Mandatory Support Requirements 

H. Desirable Features (optional) 

I. Cost Information (Unit Price Tables) 
J. Supplemental Provisions 

K* General Provisions, standard Form 32, as amended, and Additional 
General Provisions 



NOTES TO FIGURE 

Sections D. B. P. G. and H will be completed primarily by the user. The 
other sections are primarily "boiler plate** articles of information that con- 
tracting offices will supply, with minor adjustments made for the user. 
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Section B. Rep resentations. Certifications, and Acknowledgements , Con* 
tains forms that identify minority or small businesses, types of business, 
eligiblity of product, et:c. 



Section C»1 . Solicitation Instructions and Conditions . Contains in- 
structions to offerors concerning submission of bids and proposals; inspection 
of site(s) where services are to be performed, restrictions on disci osure, 
award of the contract, and forms for disclosure statement. 

Section C.2. Format Instructions . Contains guidelines for preparing the 
proposal and a description of the contract structure* 

Section D. fliossarv . Prepared by user, ADP, and contracting people. 

Section E . Special Provisions . Contains primarily contractual infor- 
mation on type of contract, terns of contract , how to handle replacement of 
equipment or repairs, standards for performance and evaluation criteria. 

User will be concerned primarily with systems life of contract and the 
specific evaluation criteria used to judge the proposals. A question that is 
frequently asked is "how is it possible to determine whether any particular 
vendor T s hardware and software can do the job?" "Benchmarks ! n Government 
procurement officers will be happy to set up a benchmark evaluation system of 
the proposals* Each of the vendors submitting a proposal must demonstrate a 
system which can do the things specified in your RFP. If they can't, they are 
disqualified. This evaluation scheme is more expensive for both your organi- 
zation and the vendor, but in large procurements, it is a small investment to 
insure a successful system. 

Section P. Mandatory Requirements. Sections include general system 

requirements, hardware requirements, software and authorizing language require- 
ments (both for system and for specific applications). User will be very in- 
volved in the development of this section. Describe functionally what it is 
that the system must accomplish (e.g. , the system will interactively deliver 
lessons to students at any terminal in the system simultaneously . ) Be spe- 
cific. Describe how the system will work for each of its users: students, 
instructors, authors and managers. Describe what reports are required and what 
they should look like. Describe the environment in which the system will be 
used. Describe essential requirements to vendors as clearly and concisely as 
possible. Let the vendors respond in terms of what hardware, software and 
authoring languages they will provide to meet your requirements* 

Section G. Mandatory Support Requirements . Will contain information on 
installation sites and delivery requirements, site planning (space, electrical, 
mechanical , communication) , hardware and software maintenance, training re- 
quirements, and manuals and publications. 

User involved here, especially on installation and site planning issues. 

Section H . .Desirable Features . In this section any features that you 
would like the system to eventually have would be listed. For example, complex 
graphics vs. business graphics. 

Section I. Cost Iflfonpation^tUnlt .Prise Tabl eal. Contains tables for 
costing hardware, software, support services, and government in-house costs. 
Instructions for determining costs and questionnaires are included. 
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Section J. Supplemental Provisions. Contains provisions regsrding con- 
tractor commitments, warranties, and representations, risks of loss or damage, 
patent and copyright indemnities, and privacy act violstion? > (These supple- 
ment general provisions in Section K, following). 

Section K« n*nprai Prnviaions . Contains provisions on making changes, 
payment for extras, variation in quantity, inspection, making payments, assign- 
ment of claims, default, examination of records, disputes, patent and copyright 
infringement, overtime compensations, etc. 

Small System s 

For those acquiring small systems or systems with limited applications, it 
may be advisable to consider researching the market, then specifying the most 
suitable hardware, software, and authoring language in an IFB. However, be 
warned, "computer people** are almost as bad, if not worse, than "training 
people" in using jargon and acronyms to describe their systems and capa- 
bilities. Learning enough to be able to talk "computerese" and to fathom the 
distinctions in the hardware and software capabilities takes considerable time 
and energy. For this reason, most organizations have found it essential to 
have a knowledgeable computer systems consultant or analyst as a member of the 
CBT development team even for small projects. 
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^ PROTOTYPE SYSTEM DEVELOPMENT 



Whv a Prototype Project? 

CBT must be bull t over time . The process of winning user support, 
developing the many new administrative procedures required , training course 
developers, and converting or developing courseware is a large task which can- 
not be accomplished all at once. Furthermore, changes in initial plans and 
minor mistakes in implementation are inevitable. Starting "small" allows the 
program manager to make corrections with a minimum of fuss or impact. In some 
circumstances it is possible to run protytpe projects simultaneously with 
planning for a large system, i.e., in advance of approval of a major system 
proposal, with the activities in each effort inter-relating. 

Project Control 

Even a "small" prototype project is a complicated undertaking. Systematic 
planning can mean the difference between a rewarding and acclaimed success 
which can result in an important building block for the evolving CBT system — or 
an embarassing failure. Figure 6-1 lists a number of elements which should be 
addressed in a project plan* This list is not exhaustive, but should give you 
a start in planning for project control. 

Level of Effort 

Once everyone i^ committed to obtaining and trying out a new system, 
materials must be developed, people trained, space prepared, equipment ordered, 
and on and on* * * The complexity of the project depends largely on the size 
of the course and the number of locations at which it will be tested* If an 
entire training course is to be redesigned to properly integrate the CAI or CMI 
material, then the tasks may be very time consuming. 

It is very difficult to give standard estimates of the time which will be 
required since so much depends on the CBT authoring language, the nature of the 
material, and the experience of the designers* The following time estimates 
have a wide variance and are at best only general indicators of the amount of 
effort that could be involved. 



Figure 6-1* Elements of Project Control 



People : 



Work : 



Honey : 



Organizational Analysis 
Job Analysis 
Position Descriptions 
Performance Standards 
Evaluation Procedures 
Award Procedures 

Performance Reporting Procedures 
Resource Allocation Procedures 
Motivation 

Training of developers and training center administrators 
Student orientation 

Administrative procedures for training deliery 

Planning Cycle 

Estimating Techniques 
Projection Techniques 

Administrative Control Element (Project Control Office) 

Development Schedules 

Release Schedules 

system Effectiveness Reviews 

Review Committees (Developer and User) 

Commi tme nt Compl iance 
Design Compliance 

Project Tracking System 

Resource Allocation Procedures 



Budget Plan 

Budget Monitoring System 
Budget Reporting System 
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The design of tutorial CAI lessons requires about 200 hours of preparation 
per hour of instruction for new CAI authors and developers, and about 100 hours 
f or experienced developers* It takes about 300 hours of time to develop an 
hour of simulated material for experienced simulation developers* Simulation 
design requires additional author expertise. If a computer based training 
program consists of computer managed instruction ( tests* etc* ) alone without 
much interactive instruction delivered directly via a computer terminal, then 
the development time for tutorial instruction required is shortened con- 
siderably. It would take an estimated 50 hours of preparatory time per hour of 
computer managed instruction (CMl) delivered* In this instance, a student may* 
for example, take a test on the computer terminal and based on the results of 
the test be referred to view a video tape and/or read a self-instruction manual 
and return to take another test* Although this form of instruction makes the 
least effective use of the flexibility of the computer as a learning resource, 
it does promote the use of individualized instruction, a necessary first step* 
In addition, more trainees can be accomodated if the training resources (com- 
puter time and course development time) are in short supply. Because of 
resource constraints, CMI may be the only way an organization can initially 
justify entry into the CBT mode of individualized instructional delivery* 

Example 

Figure 6-2 is an example of a prototype development schedule* The sample 
project schedule reflects the complexity of such projects. Designing and 
implementing the computer system is just one of the major tasks* Without high 
quality computer courseware and other training materials, well- trained people 
to operate the system, and well-equipped space, the computer system simply will 
not provide the intended results* 
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Figure 6-2, Example of a Prototype Project Development Schedule 
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2, CBT COURSEWARE DEVELOPMENT 



Development Model 

Development of computer courseware should occur within a framework for 
systematically designed individualized instruction* There are many models for 
developing instruction, no one of which is most appropriate for CBT* Although 
development models should be adopted or adapted on the basis of how well they 
fit individual organizations' styles and program needs , we recommend starting 
from the basic instructional system design (ISD)* 

Any development model used, however, should attend to the following: 

Front-end analysis : Define the learning population, assessing 
needs , analyzing tasks learners are to perform, identifying ob- 
jectives and content , sequencing objectives and content, etc. 
(This work should build on the needs assessment which preceded 
selection of CBT as the training method, Chapter 2) . 

Selection of instructional method : Identify a range of possible 
methods for delivering instruction and selecting methods on the 
basis of their costs and benefits, using information gathered in 
the front-end analysis . Methods need not be limited to com- 
puters, but can also include other appropriate media* 

Deaig fl: Pay attention to the various ways instruction can be 
designed to promote learning. Factors affecting learning such as 
the provision of practice and feedback should be identified* 
Figure 7-1 summarizes the main forms of computer-assisted in- 
struction. The forms can be combined or used independently* 

Evaluation : Attention should be given to the various methods for 
trying out and testing the adequacy of the courseware designed. 
Criteria such as effectiveness, efficiency (time and cost), etc*, 
should be identified. 

Development Team 

Before a prototype is designed or conducted, the project leader must be 
assured that all individuals participating in the prototype have the skills and 
knowledges required to perform their respective roles. CBT development is most 
successfully handled by a team configuration* The development team should 
consist of an instructional designer, a programmer and/or systems specialist 
familiar with interactive programming. Subject matter or content specialists 
can be detailed to the project for as long as it is required to provide the 
needed te chni cal information . 
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Figure 7-1 ■ Categories of Computer Assisted Instruction 



Dri 1 1 and practice . The computer presents the students with 
exercises/problems that students must perform and solve. These 
exercises may involve open ended, constructed responses (students 
compose their own answers) or pre-selected responses (students 
choose answers from a list provided). Scores can be recorded by 
the computer. Generally, the design of drill and practice exer- 
cises should follow the rules for good test construction. 

Tutorials . The computer presents information, prompts students, 
asks questions about the information presented, requests res- 
ponses, informs students about the adequacy of responses, and 
determines each student's progress path or matrix for the course. 
Tutorials can be deductive or inductive. Deductive tutorials 
present generalizations/ principles first and then require 
students to make responses involving application of the prin- 
ciples presented. Inductive tutorials begin by requesting res- 
ponses of presenting examples and illustrations of instances 
involving applications of principles and then require students to 
infer generalizations. Usually, tutorials should follow the 
rules for designing programmed instruction and/or information 
mapped materials. In order to effectively utilize the computer 
as a training tool, oonsiderable opportunity MUST be provided for 
the student to * nteract with the computer. Progress through a 
lesson may be d. ,ermined by student response, or a computer con- 
trolled branching. 

Si BU l afc ion p. The computer presents simulated Job problems or 
tasks to give the student a training experience that is as real 
to life as possible in a more controlled manner than is afforded 
by the usual on-the-job training situation. Since simulation 
training is structured and is part of a formal training program, 
it can reduce the cost and time generally required by OJT, though 
it does require a considerable amount of developmental time. If 
technical equipment is to be simulated, a high resolution CBT 
terminal screen is required. (some systems now employ a video* 
disc generated picture.) 
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Normally an instructor whose background is primarily technical should NOT 
be assigned to lead the development team. The project manager must assure the 
adequacy of staff and equipment resources and coordinate the production of 
course materials so that the team operates as efficiently as possible. The 
project manager must also ensure that the development team keeps up to date in 
the latest CBT design techniques. CBT is a fast changing area and improvements 
are frequently introduced* Professional CBT organizations can be an important 
source of such information (see Appendix C). 

Present job classification standards make it difficult to recruit persons 
with the necessary skills. For instance, the GS-235, Employee Development 
Specialist, or GS-1710, Education Specialist, series are not specific enough to 
require CBT-related skills. Recruiting in the GS-33*J, Computer Programmer/ 
Analyst, series will probably identify persons with automated data processing 
knowledges but none of the necessary training knowledges. We recommend, there- 
fore , classification of most members of the CBT development team as GS-1701 , 
Instructional Technologist. Although this series requires a bachelors degree 
it allows considerable flexibility in recruiting. Specific CBT skills can be 
required, but persons with a wide variety of backgrounds who have acquired 
those skills can meet the basic qualification requirements. 

Good quality courseware is an essential element in winning user accep- 
tance, particularly in prototype efforts when inexperienced users are likely to 
be very frustrated by shortcomings rather than having the experience peeded to 
accept thorn with good humor or a tolerant, helpful attitude. CBT, particu- 
larly CAI, requires course developers to think about training in new ways. 
The " Computerized lectures' 1 which are sometimes typical of new developers* 
first efforts can be very boring* Because they do not utilize the interactive 
capability provided by the computer. The back and forth exchange between 
student and machine is the most dynamic and salient feature of an automated 
individualized instruction program. Designers cannot afford to short step this 
process* A continuing program of in-house seminars with skilled team members 
(or outside consultants) may be necessary to bring the whole development team 
up to an acceptable level. 

Standard course prototyping and validation techniques must be supplemented 
with one additional step: the one-on-one tryout * The one-on-one try out 



consists of a course developer sitting with a typical student (not another 
already partially knowledgeable instructor or staff person — short cuts not 
permitted here I) while s(he) takes the course. The course developer should 
note everything the student does, including nonverbal behavior and thoroughly 
discuss with the student any frustrations or negative reactions. Although it 
means a little extra time and effort, this step may account for the critical 
difference between acceptance and non-acceptance of the lesson. 



8. EVALUATIOH AMD MAINTENANCE 



General 

Evaluation of training programs Is conducted at continuous as well as 
discrete and specified intervals. Evaluation checkpoints should be constructed 
within each phase of systematic training program design, development and im- 
plementation. The first step to evaluation is careful planning. The CBT 
manager and/or CBT development team should clearly specify project goals and 
training outcomes. The inter-service Instructional Systems Design (ISD) model 
previously cited could be used as a basic source of reference. The reader may 
also consult another reference by William R. Tracey, Human Resource Development 
Standarda r New York, Amacom, 1981 (See other references in Appendix A). 

Next the project manager should assure that minimum standards for the 
acceptance of each project and program goal and objective are hammered out in 
advance. These standards, as well as the project goals and/or learning objec- 
tives may have to be changed or updated as new information is obtained. 
Although this process is somewhat time consuming, it will result in the 
achievement of more qualitative products, and more assurance of optimal staff 
productivity within prescribed time and cost constraints. This is particularly 
critical when using expensive computer resources if management and user 
approval and acceptance is required. 1 Generally resource utilization is a 
critical issue in the implementation of new CBT systems. Once the benchmarks 
are determined? time tables for assessing outcomes must be prescribed. Methods 
for the collection and analysis of evaluation information should be determined 
in advance as these will affect the project's overall cost and resource re- 
quirements. 



^Ttae authors recognize that there is a considerable gap in published stan- 
dards specifically directed to the construction or evaluation of computer-based 
training materials. The IGCBT has elected to address this problem as its first 
priority for 1 983 • The availability of languages and courseware ^sr another 
separate but related concern. / 



Ongoing Project Ev aluation Components 

—Regular review of staff progress and accomplishments; check 
against project program goals and standards. 

— Problem definition {where applicable); recognition of actual or 
potential deficiency or inadequacy. 

--Collection of information to specify problem dimensions and 
potential solutions. 

--Evaluation of information. 

— Specification of the pros and cons of potential solutions. 

--Evaluation against general performance criteria or standards for 
program success. 

— Development of plan to improve the problem. 

--Establishment of priorities/ time chart 

--Determination of resources required. 

— Systematic follow up for solution and documentation progress. 

— Instiution of new or additional means of quality control (if 
indicated) . 

The above processes may (and do) take place informally via regular "problem 
solving* thought processes or discussions. However, for more complex or sen- 
sitive problems t it is strongly recommended that the CBT manager outline and 
follow all of the steps t obtain management concur ranee and document the 
results. 

Figure 1-1 in the introduction showed the major steps required to implement 
a computer-based training program. It can be seen that overall project im- 
plementation and evaluation are closely tied to both the design and implemen- 
tation of the prototype phase(s) of the CBT program. Careful documentation and 
evaluation of the prototype can provide the basis for one of your most useful 
and successful justifications for full system implementation. Costs for each 
aspect of the program should be documented as this may provide some excellent 
information for future budget estimations. 
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Training Portion of Evaluation 

This portion of the evaluation typically consists of what some define as 
internal and external phases. Others may define these phases as formative and 
summatlve. Internal, as we shall define It here, consists of the evaluation of 
the results achieved by the student after completing a computer based training 
program. This should , whenever possible* be contrasted to achievement of 
students with similar backgrounds! In comparable programs* using other means of 
training delivery (e.g. , classroom * individualized (non-automated) and CBT.) 
Achievement can be measured by classroom performance (tests) and/or by the 
student's ability to successfully perform required hands on appll cations t to 
solve case problems or to succeed at follow-on training for which the CBT 
training Is a prerequisite. 

External evaluation, as we shall define it is the phase of evaluation which 
concerns the measurement of a student *s progress in on-the-job tasks that 
closely parallel the training tasks that were completed. This later phase Is 
not often accomplished In any training program because of the demand on limited 
resources that it requires. However, one model for the collection of infor- 
mation for external evaluation is soon to be Implemented and released by the 
Internal Revenue Service, This model tests retention of the students three to 
six months following formal training to determine how much Information has been 
retained and used on the job. Information will also be collected (interview 
data and supervisory approvals) from students and supervisors In terms of on- 
the-job performance. 

Currently more training managers are attempting to more systematically 
evaluate training effectiveness and its relation to on-the-job performance. It 
should be noted here that valid systematic training evaluation Is an extremely 
costly ( though critical) process, parti cularly where there are large and/or 
widely dispersed populations. This Is why though evaluation Is always Included 
as a part of the systematic process of training design and development. It Is 
seldom implemented In a very valid or effective manner. 

However t the potential offered by the computer to conduct, analyze, and 
validate tests and other training Information Instantly makes the process in- 
finitely more affordable. Considerable benefits could result from substantial 
increases in training effectiveness. It Is recommended here that the reader 
review two additional references relating to cost effectiveness of training: 
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Greg Kearsiey, post. Bgnpfit*. and Productivity in Training Systems , Addison 
Wesley Publishing Co* , Massachusetts, 1 982 and Jesse Grlansky and Joseph 
String, Cost Ef fectiyenes a of Comouter-B ased Inst ruction in Military Training. 
IDA paper, fKl375, Institute for Defense Analysis Alexandria, VA, April, 197 9- 

Terminals at dispersed field sites or work sites could be used to collect 
on-the-job performance information, to test the readiness of employees to per- 
form increasingly complex aspects of the job, to assess ongoing training needs 
and to assure a closer reiatioship of training to job task proficiency* When 
carefully and systematically designed, training dellverd by computer can pro- 
vide one bull t-in means of evaluatl on and quality assurance of the training 
program* 



COKCLUSIQM 



In the starter kit the IGCBT members have attempted to address the most 
critical aspects of initiating and managing a successful computer-based 
training program. We have given each of these elements (depicted in Figure 
1-1) a considerable amount of thought, and we hope that they will provide some 
assistance to the training professional, particularly as s(he) is first begin- 
ning in the relatively new field of computer-based training. It is our firm 
belief that training professionals or instructional technologists who are very 
knowledgeable about the instructional design process should have the lead 
management responsibility in the design , implementation , and evaluation of 
computer- based training systems. Although subject matter experts and data 
processing personnel are very important members of the CBT design team, the 
training results that are required are best assured when each step of the pro- 
cess is designed and evaluated in a context of student needs and ultimate 
training outcomes. 

The remai nder of the conclusion of this Starter Kit is to be provided by 
you, the user or reader. Please provide your comments back to us as soon as 
possible so that we may include them in our next revision. (We've provided a 
general comments sheet in the front of this document. ) Do indicate, as 
specifically as possible, your role in the design, implementation and/or 
evaluation of the CBT system, whether you are employed in the private or public 
sector, and the size and configuration of your training population as well as 
the system you plan to configure* No information will be printed about the 
information you provide without your express written consent* We are equally 
interested in general trends* 

One area that the group would particularly like more information is the 
area of CAI standards. We have not been able to find a good overall set of CAI 
product (or lesson) standards or a CBT system standard requirements package* 
These would be of use to us to train new CBT development staff members to set 
up courseware development contract specifications, to evaluate existing CBT 
courseware, to design CBT systems, etc* This is an area that the IGCBT has 
decided to concentrate on over the coming yesr* Any suggestions, recommen- 
dations, or actual standards which you have used would be greatly appreciated* 

Finally , if you would be willing to provide assistance to other CBT 
managers, in either a formal or informal sense, please let us know. 
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APPENDIX B 



ACCESS TIME 



ADDRESS 



ALPHANUMERIC 



APPLICATION SOFTWARE 



ARCHITECTURE 



ASCII 



ASSEMBLER 



AUTHORING LANGUAGE 



AUTHORING SYSTEM 



AUTHORS 



BASIC 



BAUD 



The period of time between calling for information 
from memory and the delivery of that information. 
A disk storage is generally faster than tape 
storage. 

Designates where information is stored within a 
memory device. 

The computer symbols, letters ( A-Z) and/or 
numerals (0-9) » and/or special punctuation, mathe- 
matical, or graphic symbols. 

The software required to perform the user appli- 
cations described in the functional requirements 
section. 

The internal, preset arrangement or organization 
of a computer that determines how the computer 
operates . 

American Standard Code for Information Inter- 
change . A Binary number code for letters , 
numbers, symbols, etc., accepted as a standard by 
the industry. 

The program that converts English in machine 
language (binary format) . 

Programming language with codes specifically 
designed to handle major courseware needs such as 
response Judging. 

Prepackaged software designed to help authors 
create courseware without elaborate programming. 

Writers who use the CBT language processor to 
create computer courseware, for either CMI or CAI 
applications. 

Acronym for Beginners All- Purpose Symbolic In- 
struction Code. Using common English words and 
mathematical symbols to perform arithmetic and 
logical operations in the computer to solve 
problems. 

Date with which one device transfers information 
in one second. 1500 Baud is equivalent to about 
150 characters per second. It is stated as bits 
per second ( bps) • 



BINARY CODE 



A code that uses zeros and ones for data. 
10110011 may represent the letter C, 



BIT 



Binary Digit. The smallest unit of digital in- 
formation thought of as representing: a yes/no 
choice or whether a circuit is on or off, stated 
as a zero or one . 



BPS 

BRANCH 



Bits per second. 

An instruction when met in a program makes a move 
to another part of the program deviating from the 
normal sequencing of the program. Branch state- 
ments are one of two types: conditional statement 
(command IF . . . THEN) or unconditional statement 
(command GO TO) . 



BUG 



Error--be it a programming error or equipment 
malfunction. 



BUS 



Set of wires that allows data to be sent generally 
with microcomputers bidirectionally. 



BYTE 



CASSETTE RECORDER 



CHIP 



CODE 



CODING 
COMMAND 



A group of binary bits. Eight bits is equal to 
one byte. It takes 8 bits to form one character; 
so one byte is generally equal to one letter, 
number, or graphic symbol. 

A device through which information is stored. The 
information is put onto the cassette as audio 
signals. The cassette information is stored 
sequentially and therefore a slower format for 
storage of information. 

The heart of a microcomputer, on which thousands 
of electronic elements are implanted. This piece 
of silicon, a microprocessor, contains all the 
ci rcui try to carry out the many compute r oper- 
ations. It is created through a photographic 
etching process. 

The relationship between bits and a set of charac- 
ters, alapbabet letters, numbers or graphic 
symbols. Each character has its own bit code 
(zero or one binary representation) . The most 
commonly used code is ASCII. 

Developing a set of computer instructions. 

An instruction given to the computer from an input 
device . 
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COMPATIBILITY 



COMPILER 



COMPUTER 



COMPUTER-BASED TRAINING 
(CBT) 

COMPUTER- ASSISTED 
INSTRUCTION (CAI) 



There are two types of compatibility : software 
and hardware. Software compatibility refers to 
the ability to run programs on a variety of 
computers. Hardware compatibility means that 
various components ( printers , disks , keyboards , 
etc.) may be connected directly. 

A program that translates machine language into 
basic (everyday) English or other computer pro- 
gramming language and vice versa. 

An assembly of switches. It can be programmed to 
process information. Computers come in three 
categories. A mainframe can execute up to ten 
million instructions per second and costs from 
$100,000 upward. A mini-computer can perform one 
million calculations in a second and costs from 
$10,000 to $100,000. A micro-computer, which is 
what a home computer is, can carry out 250,000 
instructions in a second and will cost between 
$100 and $10 t 000. 

An overall term referring to the use of computers 
in the training process. 

One application of CBT which involves an on-line 
interactive process between a student and a com- 
puterized delivery system in which the computer 
assumes a direct instructional role. 



COMPUTER- MAN AGED 
INSTRUCTION (CMl) 



Another application which refers to the use of a 
computer to manage students f progress through a 
training course. A CMI system supports ins true* 
tions by selecting, presenting, and scoring tests, 
recording student progress data, providing feed- 
back on student drill and practice and test per- 
formance, and prescribing use of various learning 
resources (such as video tapes , textbooks, slide- 
tapes, reference materials or field visitations). 



COMPUTER SYSTEM 
CONTRACTOR 



All hardware and software necessary to meet the 
requirements. 

One who prov ides , supplies , and renders serv ices 
in accordance with the terms and conditions of a 
contract . 



COURSEWARE 



Instructional material for student or instructors 
for CAI and CMI applications. Separate courseware 
is available for each course managed by the sys- 
tem. The courseware packages for each course may 
include lessons, modules, units, chapters, and 
objectives. 
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S3 



CPU 



CRT 



CURSOR 



DATA 



DEBUG 



DIAGNOSTIC ROUTINE/ 
PROGRAM 



Central processing Unit. Also known as a micro- 
processor! this is the main switching circuity, 
the brains of a computer. The CPU holds the 
silicon chip. ^ 

£athode-£ay Xube. Its display screen is like the 
one in your TV. Also known as a monitor. The CRT 
is a common output device. 

A mark on the display screen that shows where the 
next character will appear. 

The information given to or received from a 
computer. 

Process of finding, locating, and correcting 
errors in a program. 

A program that will check out the hardware and 
peripherals for incorrect information and 
breakdown. 



DIGIT 

DIRECT MEMOKY ACCESS (DMA) 
DISK (DISC) 



DOCUMENT 

DOS 

DOTMATRIX 
DRIVER 

DUMB TERMINAL 
DUMP 



A zero or one in the binary system. 

A technique to move data rapidly from the micro- 
processor to a storage device (i.e. disk)* 

Magnetic coated material in a 5" or 8" record-like 
shape on which information and programs are 
started. The information is stored randomly and 
therefore faster than cassette storage. Sometimes 
called diskettes or floppy disks. 

A written description of a piece of software or 
program or piece of equipment. 

Disk Operating System. A set of programs and 
instructions that permit interaction between the 
diskettes and the microcomputer. 

A method to generate graphic characters by using 
dot patterns. A 5" x 7* dot matrix is a common 
example . 

A program that controls the peripheral devices and 
how they interact with the CPU. 

A terminal that acts as an input/output device 
only. 

Transfer (or print out) memory from one area to 
another within a computer system. 
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EDITOR 

ERROR 

EXECUTE 

EXTERNAL STORAGE 

FILE 

FILENAME 

FLOWCHARTING 

FORTRAN 
FREQUENCY 

FULL DUPLEX 

GIGO 



GLITCH 



GRAPHICS 



HARDCOPY 



HARDWARE 



HEAD 



IC 



A program that allows a person to change or modify 
a program. 

The difference between the actual response of $ 
problem or program and desired response. 

The running of a computer program. 

Peripheral device for storage (i.e. tape or disk)* 

Collection of related data. 

Number or letter characters to identify a file , 
collection of data. 

A programming technique using shaped blocks indi- 
cating the direction and sequence of operations in 
a program. 

Science-oriented program language. 

Date at which anything occurs and recurs, measured 
in cycles or hertz per second. 

Reception and transmission of information at the 
same time • 

Garbage in, garbage out. If incorrect information 
is put into the computer, the output will be 
misinformation. 

A burst of line noise that can cause a computer to 
fail or crash. 

Characters used to create figures, shapes, and 
forms on the CBT or printer. 

Data or information printed on paper distinguished 
from printed information or a temporary image on 
the computer's screen. 

The physical equipment that goes into a computer 
system. The essentials are the central processing 
unit, the memory and an input-output device. 

On a cassette recorder it is the area that reads 
information from a pre- made cassette tape. 

Integrated Circuit. A plastic or ceramic body 
with numerous leads extending from it. The 
silicon body protects the chip. The leads permit 
electrical connection of the chip to other com- 
ponents. The word "chip* may also be called a 
m i cro pro ce ssor . 
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IMMEDIATE ACCESS STORAGE On-line, random access storage with update 
(IAS) capability. 



INPUT 



INSTRUCTION 



INTERACTIVE 



INTERFACE 



INTERNAL STORAGE 



INTERPRETER 



I/O 



KEYBOARD 



KILOBYTE 



LANGUAGE 



LETTER QUALITY PRINTER 
LOAF 

MACHINE LANGUAGE 



The information that you enter into your computer; 
what you tell the computer to do. A keyboard is a 
common instrument for input. 

A group of bits or a command that will make the 
computer perform a specific operation. 

All personal computer systems are interactive , 
allowing for two- .y communication while it is 
being used. 

Electronic circuit that connects the CPU and a 
peripheral device, disk drive, etc. , permitting 
the flow of data back and forth. 

Memory system inside of the computer, rather than 
disk or tape storage. 

A program used to translate various computer' 
languages. 

Input and output of data and Information of a 
computer system; examples are: keyboard, floppy 
disk drive, printer, cassette recorder, modems, 
graphics tablets. 

The computer version of a typewriter. When you 
depress the keys, which are really switches, data 
is fed into the memory. 

The standard unit of the memory. One kilobyte 
comprises 1,024 bytes. (An 8K home computer, then 
stores 8,192 bytes of 65,536 bits.) 

A format that allows a programmer to communicate 
more efficiently with a computer where commands 
will give requested actions, BASIC being one of 
the most popular languages. 

A printer which produces letter quality output 
utilizing either single sheet or multi-part 
form sets. 

Putting information into the computer's memory. 

A computer programming language that has its 
instructions in binary, octal or hevadecimal 
format. 




MAIN MEMORY 



That memory which Is directly accessible to the 
computer. In a microcomputer, main memory Is 
referred to as RAM or ROM. 



MEMORY 



MEMORY CHIP 

MENU DRIVE SOFTWARE 



MICROCOMPUTER 



MICROPROCESSOR 



MODEM 



MODULATOR 



MONITOR 



MOTHERBOARD 



The lntergrated circuits of a computer which 
stores information and instructions. Each piece 
of information has a specific location assigned to 
It within the computer's memory (see also 
STORAGE). 

A chip on which data Is stored as electrical 
charges. 

The availability of options by which a user Is 
prompted to specify which functional operatlon(s) 
s(he) desires to perform within a given appli- 
cation. 

Usually included in a microcomputer is the micro- 
processing unit, keyboard for entering data, and a 
cassette recorder or disk for storing programs t 
and a monitor. .1 hardware configuration usually 
acquired in one of three ways: constructing 
several components from individual electronic 
parts; connecting several already* constructed 
components; or by purchasing a unit with built-in 
components. The microcomputer records infor- 
mation! processes it, puts it into meaningful 
terms 1 communicates it, stores it, and retrieves 
it. 

An integrated circuit that executes instruction 
inside the microcomputer. 

A device that makes computer information transfer 
over phone lines possible. It changes the tone to 
signals understood by the computer. 

A device called an RP modulator to permit a stan- 
dard television to act as a video display unit. 

A video display unit which uses a cathode ray tube 
to generate characters with high lines of reso- 
lution, permitting very small characters to be 
projected on the screen. 

The central board or boards inside the micro- 
computer Interconnects the various chips, and 
allows the interface between memory and peripheral 
devices. 



NOISE 



Inaccurate data transmission. 




NONVOLATILE MEMORY 



Memory that holds data even after the power has 
been shut off. ROM is nonvolatile. 



OEM 



OUTPUT 



PERIPHERALS 



PERFORMANCE REQUIREMENTS 

PERSONAL COMPUTER 

PLOTTER 

PORT 

PRINTER 



PROGRAM 



PROMPT 



RAM 



READ 



Original £quipment Manufacturers are compani es 
which assemble components manufactured by others 
and sell them as systems. 

The Information the computer displays (on the 
CRT) t prints, or transmits after It ocesses your 
Instruction (Input). 

The accessories, or options, that can be added to 
your basic computer, such as a printer, a disc 
drive, or Joysticks for playing video games. 
Peripherals can be more costly than the basic 
computer. 

This term means the performance ( capacity) re- 
quired by the proposed system to accommodate the 
specified workload and files. 

A microcomputer designed for Instructional uses, 
entertainment, or personal recordkeeping. 

A device that draws on paper two dimensional 
shapes and designs. 

The connection spot on the microcomputer where 
input and output devices can be connected. The 
most common part is an RS232 plug. 

A peripheral device that collects output data from 
the microcomputer and prints it on paper. Prin- 
ters are defined as Impact or non-impact. Impact 
printers strike the paper by a ribbon^like a type- 
writer. Non- impact printers form characters by 
electrical charges, or spraying ink. 

A unit of discrete computer instructions which 
causes a computer to pef orm a sequence of oper- 
ations. A program may contain "routines* , "sub- 
routines*, and "modules*. 

An instruction or a sign from the computer that 
tells you what to do. For example, WAIT or 
BEGIN. 

Jiand om^JLc cess Hemory. This is the data that you 
can call up from the computer's internal storage. 
You can read it, change it t or erase it. 

The act of retrieving from memory or from an 
input/output device. 
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REGISTER 



A temporary storage device located in the micros 
processor which holds computer bits. 



RESPONSE TIME 



ROM 



HUN 

SMART TERMINAL 
SOFTWARE 



SPIKE 

STORAGE CAPACITY 



STORAGE DEVICE 

STORE 
TAPE 



TERMINAL 



TRACK 



The time interval required for the microprocessor 
to respond to an instruction or input device. 

Read Only Memory. A circuit where data or in* 
struct! ons are programmed at the time of manu- 
facture. It allows the microcomputer to talk to 
us in BASIC. It cannot be erased during normal 
operation. 

Jargon for execute. 

A terminal that has the ability to process data 
and function as a computer in addition to being an 
input/output device for a main frame computer. 

The prorams and documentations to be run on the 
physical equipment (hardware) . The three current 
formats of software storage are tape (cassette) , 
diskette , and solid state. The microcomputer 
loads the software into its memory to utilize the 
software programs. 

A temporary sharp increase in signal or voltage. 

The quality of bytes a storage unit can hold. A 
diskette is said to have *l8-K (48,000 bytes), has 
approximately ^8,000 characters, letters, numbers, 
spaces, or symbols. 

A peripheral device that stores information, i.e., 
tape or disk. 

Placing information in a storage device. 

The most common microcomputer 1 tape is magnetic, 
such as cassette tape. Magnetic tape is stored in 
electrical charge patterns that are equivalent to 
what we know as letters, numbers, symbols, etc. 

A peri pheral dev i ce whi ch al lows human communi- 
cation with a computer. When it is called a 
terminal, it is frequently tied to a main frame 
computer. 

Where electrical charges or characters are stored 
on diskettes. Some diskettes have up to 70 or 
more tracks on one diskette. 
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TURN KEY SYSTEM Computer system packaged with proper hardware and 

software to be able to perform designed appli- 
cation upon turning it on. Many word processing 
systems are sold as turn-key systems* 

USER FRIENDLY SOFTWARE The ef.se of use of vendor-supplied software 

designed to facilitate data entry, screen for- 
matting t data base definition, inquiry, and report 
generation* This software shall be of such 
sophistication that a user may learn to design 
screen formats, make data base queries, and 
generate reports ( including editing and sub- 
totaling) with a minimum of formal training* 

VIDEO DISPLAY UNIT (VDU) A part of a microcomputer similar to that of a 

telev ision where information is placed on its 
screen* 

VOLATILE MEMORY The memory device when the electricity is turned 

off and the information is not retained* RAM 
memory is volatile* 

WINCHESTER DISK A high speed, sealed, large-capacity , magnetic 

storage system for microcomputers* 
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FAfiT 1 - ORGANIZATIONS. ASSOCIATIONS. CONSORTI A AND NETWORKS 



ACM Association for Computing Machinery 

1133 Avenue of the Americas, New York, NY 10036 

— ES3 is an ACM subcommittee on elementary and secondary schools 

— SIGCUE - ACM's Special Interest Group for Computer Users in 
Education Publishes the SIGCUE BULLETIN quarterly 

ADCIS Association for the Development of Computer-Based Instruction 
Western Washington State College, Bellingham, WA 98225 

AECT Association for Education and Communications Technology 
1126 16th St. NW t Washinton, D.C. 2036 

AEDS Association for Educational Data Systems 

1201 16th Street, NW t Washington, D.C. 20036 

AERA American Educational Research Association 
1126 18th St. NW, Washington, D.C. 20036 

ASTD American Society for Training and Development 

Suite 305, 600 Maryland Ave. SW , Washington, D.C. 20024 



CONDUIT Jim Johnson and Hal Peters 
P.O. Box 338, Iowa City, I A 



52240 



^-CONDUIT is a source of information on instructional materials 
and instructional applications for computers and current 
research projects. 



ICCE International Council for Computers in Education 
c/o Computing Center 
Eastern Oregon State College 
La Grande, OR 97850 

MEAN Microcomputer Education Applications Network 

Suite 800, 1030 15th Street, NW, Washington, D.C. 20005 

MECC Minnesota Educational Computing Consortium 

Publications Department, 2520 Broadway Drive 
Lauderdale, MN 55113 



—MECC has studied the use of microcomputers for both instruction 
and administration 



NSPI National Society for Performance and Instruction 

1126 Sixteenth St. NW, Suite #315, Washington, DC 20036 

NWREL The Northwest Regional Educational Laboratory 
Computer Technology Program 
710 SW Second Avenue, Portland, OR 97 2 04 



ERLC 



PART 2 - CB T VENDORS LIST 



The following list shows both hardware and software vendors, some of whom 
are directly involved in packaging CBT systems. Some have supplied other hard- 
ware or software to others for use in a CBT system. This list is by no means 
exhaustive , but t represents vendors with which some of our members have had 
contact with, have seen demonstrations of or received responses from on various 
CBT system proposals. No inference as to the quality of the vendors listed or 
their products is made. 



Apple Computer 
10260 Bandley Drive 
Cupertino, CA 95014 



Advanced Technology Applications 
4296 Tambor Court 
San Diego, CA 92124 



Bell and Howell 

Audio-Visual Products Division 
7100 N. McCormick Rd. 
Chicago, IL 60645 



Boeing Computer Services Co. 
Education and Training Division 
P,0. Box 24346 
Seattle, WA 98124 



Calculon Corporation 
federal Systems Group 
1 30 1 Piccard Dr. 
Roclcville, MD 20850 



Control Data 
Box 0 

Minneapolis, MN 



55440 



Digital Equipment Corp* 
12 Crosby Drive 
BU/E33 

Bedford, MA 01730 

Engineering Research Assoc. 
1517 Westbranch Drive 
McLean, VA 2210 

Global Information Systems 
Technology, Inc. (GIST) 
201 West Springfield, Suite 1008 
Champaign, IL 61820 

Gould SEL 

2901 West Sunrise Blvd. 
Fort Lauderdale, Fl 33310 

Hazeltine Corporation 
7680 Old Springtiouse Road 
McLean, VA 22102 



£mcco t Inc. 
RR #3 Plaza Del Sol 
Village of Oak Creek 
Sedona, AZ 8£53l 

General Technical Corp. 
1101 No. Fulton Ave 
Evansville, IN 477102 

Goal Systems, Inc. 
P,0. Box 29481 
Columbus, OH 43229 



Group Technology Ltd 
P.O. Box 87 
Check, VA 24072 

Hughes Aircraft Co. 

Corp. Marketing RFP Control 

P.O. Box 92996 

Los angeles t CA 90009 



ERIC 



IBM 

Data Processing Division 
10**01 Fernwood Rd. 
Bethesda, KD 203** 

Info III 

21250 California St. 
Woodland Hills, CA 91367 



Logicon, Inc 

P.O. Box 80158 

San Diego r CA 92138 



McDonnell Dougles, Inc. 
P.O. Box 3020** 
Denver, CO 80230 

Modcomp 

P.O.Box 6099 

1650 West McNab Rd. 

Ft. Lauderdale, Fl 33310 

Phoenix International 

P.O. Box **5165 

Los Angeles r CA 900**5 

SAI Com systems, Inc. 
2801 Camino Del Rio South 
San Diego, CA 92108 

Syscon Corporation 
1901 N . Fort Myer Dr. 
Rosslyn, VA 22209 

Texas Instruments, Inc. 
P.O. Box 225012, M.S. 23 
Dallas, TX 75265 



HAT 

20010 Gentry Blvd 
Suite 100 

Geroatown, HD 2087** 

Institute for Advances Technol 
Control Data Corp. 
6003 Executive Blvd. 
Rockville, MD 20852 

HcDonall Douglas Corp. 
1150 17th St., N.W. 
Suite 500 

Washington, D.C. 20036 

Middle Sex Research Center 
3^13 1/2 H St. N.W. 
Washington, D.C. 20007 

On-Line Systems, Inc. 

** Professional Dr., Suite 119 

Gaithersburg, MD 20760 



Regency Systems 

1610 Interstate Drive 

Champagn t IL 61820 

Symtec, Inc. 
P.O. Box 462 
Farmington, HI 48024 

Systems Research Laboratories 
2800 Indian Ripple Rd. 
Dayton, OH **5440 

Vision Inc. 

1401 Avenue of the Stars 
Suite 585 

Los Angeles, CA 90067 



WICAT 

2103 Gunnell Farms Dr. 
Vienna, VA 22180 



This lists various autl >ring languages, authoring systems, instructional 
delivery systems and, where known, the marketing source. Again, this list is 
not exhaustive, but includes systems we are somewhat familiar with. 



ASSET 

AIS 

CAN-8 

cohere 

DAS 

eduCAItor 

Ghostwriter 

IIS 

LOGO 

PASS 

Phoenix 

PILOT 

PLATO 

R2 

Scholar/Teach 3 

Simpler 

TERAK 

TICCIT 

TUTOR 

WISE 



Univac 

McDonnell Douglas 
Honeywell 
Emcco, Inc. 
DEC 



IBM 

MIT/BBN 

Bell and Howell 
Goal Systems 

Control Data Corporation 
Regency 

Boeing Computer Systems 
ModComp 

University of Utah 
Hazel tine 

Control Data Corporation 
Wicat 



APPENDIX D 
AGENCY PROGRAM SUMMARIES 



DEPARTMENT OF LABOR 

The U.S. Department of Labor's Employment service is developing CBT appli- 
cations for several purposes: 

Staff Train ing. A state agency is currently piloting for DOL a CAI module 
for training interviewers in small offices at remote locations- This course 
was developed with Bell and Howell's PASS and BASIC authoring languages. Apple 
microcomputers are being used t but the course will be adapted for delivery on 
the IBM personal computer and other microcomputers. 

Teste s- A DOL-funded test center in North Carolina is using Apple to 
score and interpret paper and pencil aptitude tests • In addition, piloting 
work is being done using a special high resolution CRT and the CTR microcom- 
puter to administer and score the General Aptitude Test Battery (GATB). 

Vocational and Career Guidance . Programs are being developed to provide 
vocational counseling through interactive CAI techniques on Apple with BASIC 
authoring language. Programs for persons entering the workforce will differ 
from those for experienced workers. 

For additional information, contact: 

Walter Martin 

U.S. Employment Service 

601 D Street, N.W., Room 8028 

Washington, D.C. 20213 

FTS 3T6-&908 

BUREAU OF THE CENSUS 

The Census Bureau started using CBT for traj ning ADP personnel in 1 974 . 
Since the Bureau primarily used UNIVAC equipment for its general ADP needs , it 
acquired ASET, UNIVAC's Author System for Education and Training for CBT usage. 
At the outaet t Census was a UNIVAC test site for the development and enhance- 
ment of the ASET language. Under this arrangement Census got the use of ASET 
at no cost plus increased support from UNIVAC. 

From 197^ to January 1979, efforts were focused on the development of CAI 
courseware to train computer operations personnel, programmers, and other users 
of the ADP system. Sone couse development is continuing. 



The CBT effort is currently a part of the Instructional Resources and 
Management Section of the Computer Operations Division's User Training and 
Information Branch. The section is responsible for all forms of self- 
instruction; CBT is but one variety. 

The impact of the CBT work at Census goes far beyond that location. Not 
only has the Bureau acted as a test site for the development of the ASET lan- 
guage , the courseware developed there has been distributed through the UNIVAC 
users' organization and the system staff of the Computer Science Center , 
University of Maryland, to many different UNIVAC locations. 

For further information contact: 
Karen H. Defazio 

User Training and Information Branch 

Systems Support Division 

Room 3136 t FO-3 

Bureau of the Census 

Washington, D.C. 20233 

FTS 763-5562 

FEDERAL AVIATION ADMINISTRATION 

The Federal Aviation Administration began experimenting with PLATO CBT in 
September 1 974 with the installation of four terminals that used the University 
of Illinois PLATO facility. The project has investigated a variety of CBT 
applications. In particular, many lessons have been developed to supplement 
Flight Standard courses . These lessons simulate flight inspection maneuvers, 
procedures, and the functions of avionics equipment. In developing these 
lessons, a team approach was used, combining the efforts of a subject matter 
expert, a lesson design specialist, and a PLATO computer author. This inter- 
disciplinary approach proved to be the most successful approach to the develop** 
ment of CBT lessons. 

Two lessons have been jointly developed with American Airlines. In this 
arrangement FAA provided hardware support, and American Airlines provided staff 
power to develop the lessons. 

The assessment of the cost-effectiveness of these early efforts was con- 
sidered promising enough for FAA to expand the project. Four additional R^ATO 
terminals were obtained. These used the Control Data Corporation PLATO 
computer mainframe. 



In another demonstration project PLATO terminals were installed in the 
seven Flight Inspection Field Offices (FIFO). These terminals were used to 
provide training usually given in a two-week course at the Aeronautical Center. 
The PLATO instruction included simulations of an inflight, on-board computer, 
CAI t and CMI. The favorable results of the demonstration project led to rein* 
stallation of the ELATO Terminals. Additional courseware is being developed 
and delivered via these terminals. 

The accelerating use of CBT in the aviation industry was recognized by FAA 
management. As a result, FAA initiated a comprehensive study of its training 
requirements and hov CBT might be used to cost-ef f ectiv ely fulfill some of 
these requirements. The study was a Joint effort of Training Programs Division 
and the Office of Management Systems. 

This study followed the approach specified by the FAA internal regulations 
on the acquisition of ADP systems. The project specified training require- 
ments, identified feasible alternative CBI systems that will meet the training 
requirements, evaluated the costs of feasible alternatives, and compared the 
findings with the existing training system. The study results were favorable, 
and an interim system proposal, written and approved in April 1979, was im- 
plemented. This interim system includes a total of 70 PLATO terminals located 
at the FAA Academy and at Airway Facility sectors. Nine courses for elec- 
tronics technicians, representing about 20 percent of the Airway Facilities 
training workload, will be handled by the system. An expanded system is under 
consideration involving the lease of 350 terminals and the extension of service 
to air traffic control centers. 

For additional information, contact: 
John Buck 

Technical Training Branch, APT-3M0 
Federal Aviation Administration 
600 Independence Avenue, N.V. 
Washington, D.C. 20591 
FTS M26-8865 

VETERANS' ADMINISTRATION 

The Veterans' Administration has developed substantial CBT courseware for 
ultimate use in training users of VA's new data processing system called 
TARGET. This system is a nationwide computer network for manipulating and 
maintaining veterans' benefit data. The system is used for inquiry, updating 



records, and producing benefit payments. Honeywell computers are located in 
each of three VA regional data processing centers. These computers support 
about 3000 terminals and 500 printers nationwide. 

About 10, 000 VA personnel work with this sy tern and were trained initially 
in its use . These employees are distributed in 58 offices among the three 
regions throughout the country . After experimenting with various modes of 
training, VA concluded that the most effective way is to train on the terminal 
equipment that employees will actually be using on the Job. 

The CAI courseware was dev elope d by the User Education Div ision of the 
Benefits Delivery Systems Support (BDSSS). At the time the CAI courseware was 
developed, this division consisted of seven permanent staff and about seven 
detailed subject matter experts who assisted in lesson development. Main- 
tenance of CBT courseware is now accomplished by BDSSS Management Staff, which 
was merged with the User Education Division. Three staff members 1 time is 
devoted to maintenance of CBT courseware. 

VA did its lessoo development using SPEAR, a courseware language developed 
uniquely for the system. SPEAR (Subsystem for Programed Education and Review) 
is written in COBOL 74 and was designed especially for the VA 1 s claims pro- 
cessing system. 

There are currently about 40 CAI lessons available for use in 58 regional 
offices. Since all regional offices have completed initial training for in- 
stallation of the system, the program is currently being used for replacement 
training. There are no plans for expansion of CBI. 

For additional information, contact: 

Leslie Schuman 
810 Vermont Avenue, N.W. 
Veterans 1 Administration 
Washington, D.C. 20420 
FTS 389-5375 

INTERNAL REVENUE SERVICE 

IRS has 75,000 employees, or whom some 5000 are located in Washington, 
D.C, and the balance in seven regional offices, ten service centers, 58 dis- 
tricts, and 200 local posts of duty . Responsibility for training is cen- 
tralized in Washington under the director of the Training and Development 
Division, who reports to the Assistant Commissioner for Human Resources. Most 
training is performed at nine regional training centers, except for special 
agents, all of whom receive training at Glynco, Georgia. 
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Revenue officers, revenue agents, tax auditors , special agents, and tax** 
payer service representatives receive basic and advanced technical training to 
qualify them to perform their jobs* They are also exposed to periodic updating 
training* Clerical and data processing employers also receive technical 
training. Managerial and executive training is provided at all levels from 
first- line supervisor to senior executive service * Over 500 courses are 
taught, 95 percent of which are developed in-house. A typical technical 
recruit training program consists of one week of preclassroom orientation at 
the post of duty, seven weeks of classroom training at a training center, and 
six weeks of on-the-job training (OJT) at the post of duty. Preclassroom 
training is supported by sel ^instruction booklets, videotapes, a leader's 
guide, and a trainee progress system record. Classroom training is conducted 
in traditional manner with student texts, an instructor's guide, detailed les- 
son plans, videotapes, and siide tapes. 

CBT has been in use since 1 975 , beginning with the purchase of one fLATO 
terminal from the University of Illinois. The system was expanded to four 
terminals and then to 19* For the year ending June 30, 1 982 , costs of CBT 
operations were $248,000 and benefits were calculated at $789,000 for a net 
benefit of $544,000. 

The present computer system (PLATO) primarily supports assembled classroom 
training operations through the following applications: 

1. Centralized test banks of test questions for most recruit and 
some advanced technical training programs. 

2. Scheduling of interregional and regional classes. 

3* Data analysis of questionnaires and evaluations. 

4 • Compiling of training course requirements and budgeting for 
regional training needs and headquarters work plans. 

5. Electronic mail. 

6 * Incidental use of CAI, utilizing PLATO catalog and self- 
authored courses, for experimental purposes. 

Beginning in 1981 the IRS Training Division has been working on redesigning 
their instructinal system model to improve the effectiveness and efficiency of 
IRS training. The new design model calls for decentralizing training to about 
200 si tes nationwide. Training which can effectively be done in a self- 
instructional format (about 50$ of all training) can th^n be moved to local 



PODs, thereby saving travel and perdiem costs. To maintain control and in* 
crease flexibility in lesson delivery a computer system will be needed to per* 
form CMI and CAI functions, as well as other data management and communications 
functions * 

The PLATO system which IRS currently uses could perform most of the new 
requirements | but would be astronomically expensive. Other methods for meeting 
our needs are being explored* 

An RFP was released in February 1 982 to lease a prototype CBT instructional 
system for a one region {7 sites) test of the new model with an technical 
recruit course, Revenue Agent I. None of the vendors submited responsive pro- 
posals at that time. A test of the new course material is now scheduled for 
January 1 983 using the PLATO system at 2 sites. Work continues on writing a 
new RFP for the nationwide computer system. 

For futher information on the current system, contact: 
Joyce Davenport 

IRS-Training and Development Division 
2221 Jefferson Davis Hgwy t Room 306 
Arlington, VA 20202 
(703) 557-2M56 

For information on proposed new system, contact: 

Lori Gillespie 

(same address and phone as above) 

U.S. SENATE 

The U.S. Senate Computer Center provides computer services to each senator 
through Hazeltin > 1510 terminals located in the senator f s Washington office and 
in up to two state offices* Currently a total of 109 state sites* The center 
is also responsible for training senatorial staff members in the use of these 
computer services. 

The central processor is an IBM 168 with attachments. Using IBf^s Inter- 
active Instructional System (IIS) software t the center has developed 10 CAI 
lessons which require an average of 20 hours to complete, for training users of 
the Correspondence Management System and the Legislative Information System* 
These lessons are replacing and supplementing classroom training* Seven 
additional lessons are under development* The staff that prepares CAI lessons 
consists of four instructional writers and one IIS programmer/administrator* 
Some 300 hours of development time (from task analysis through evaluation and 
revision) is required for each lesson hour, but this ratio is improving with 
experience ♦ 
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A formal evaluation of the program has been completed, which found the CAI 
instruction to be as effective as classroom Instruction and preferred by 90 
percent of the trainees. Student training time has been cut by 40 percent. 
Travel of training staff has been cut by 50 percent. Development coats are 
paid off In approximately two years* 

For further irif ormation, contact: 

Harlan Saunders 
U*S. Senate Computer Center 
400 N. Capitol Street 
Washington, D*C. 20510 
(202) 22*1-3127 

ARMY RESEARCH INSTITUTE FOR THE BEHAVIORAL SCIENCES 

The Basic Skills Instructional Systems Technical Area of the U . S. Amy 
Research Institute conducts research on the application of computer technology 
to training problems. Several projects have been initiated to Improve 
soldiers' basic skills. The computers and Instruction used in these efforts 
range from PLATO with approximately 50 hours of basic skills lesaons to a 
hand-held device being designed to teach and drill military vocabulary* Com- 
puters are being used for Information purposes as in AREIS t the Army Education 
Information System. AREIS Is a micro computer-* based Army career and educational 
system designed to support Army Education Center counselors* Thaee projects 
and others involve research on the design » development, and evaluation of 
computer-based systems which support the Army's training requirement* 

For further Information contact: 

Dr* Melissa Berkowltz 

U.S* Army Research Institute for the 

Behavioral Sciences 
5001 Elsenhower Ave. 
Alexandria, VA 22333 
(703) 27*1-8876 
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TAILS OF CURRENT C1T ACT1V1TBS 1H FEDERAL TRAINING tcon't J 
Prniram Statue 1 net rurt ionel Strategic* llnrtlvare 



i 



i 



Subject Arc** 



7$ 



Operet ing 
System 



Sp f t ware/ P rograan i ng 
Au t hor i ng Langu age 



Labor Department, U. S + 
Employment Service 

Labor Department Employ- 
(*en: Standard* Adminie. 

HIIS, National Inatitue 
of Health 

HHS, tUter Hilt 



Center 



OPM, Inet ruction a I Sytteem 
and Technology 

Senate Computer Center 



Social Security Ad* in 



veterene Admtntetretion 

(1) Kateriala Develop- 
vent 

(2) Onl inc Search 
As nistence 



Data gathering 
ekille 



Procurement 

Uacr on line catalog 
training 

Dentel~orel petr- 
ology 

lomputer literacy 
FORTRAN COBOL 

Word Proceeding, 
data entry and 
retrieval 

Technical, man- 
agerial and common 
needa 

Claims proceeding 



Bell-tfcutU 
D0S-*rdif*d 



IBM 
IBM 



IBM 3278 
303: HP 2626AI 



Apple llfcpple 11 



PLATO 
Etc, 



MVS 



Apple DOS 



IBM 370 

Model 

1156 

DEC 
20/20 



HateltLiw CISC 



PASS 



COBOL 
PL/1 ALC 

Liater Hill Pilot 
Applesoft , BASIC 

TUTOR t etc. 

IBM Couraevriter 



Honeywel 

%/eo 



510 



Concept 
100 



tbreywell 

7760 



OS/V52 



IDG0 



, LISP 



DM IV 



SPEAR 
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APPENDIX F 



LIST OF TOflftT ORGANIZATIONS AMD CONTACTS 



Organiza tions on Appendix E Table of CBT Activities 



U.S. Department of Ariculture 
APHIS t PPQ 

Professional Development Center 
195 Thomas Johnson Drive 
Frederick, MD 21701 



William 0. Wade, Jr. 
(301) 663-8585 
FTS 933-1270 



Air Force Human Resources 

Laboratory 
Lowry Air Force Base 
Denver, CO 80230 



William A. Nunns 
(303 ) 370 - 27 7 5 
AV 926-2775 



U.S. Air Force 

Air Training Command 

Randolph Air Force, Base TX 78150 



Frank Schuf letowski 
(512) 652-3274 



U.S. Army 

Missile £ Munitions Center School 
Redstone Arsenal 
Huntsville, AL 35897 



John P. Carrigy 
AV 746-1820 



Comptroller of Currency 
Washington, DC 20219 



James R. Wingrove 
(703) 389-4234 



Federal Aviation Agency 
Technical Trailing Branch 
APT-310 

800 Independence Ave 
Washington, D.C. 20591 



John Buck 
Barbara Wright 
(202 ) 426 - 8865 



Federal Emergency Management Agency 
National Fire Administration 
1825 S. Seton Ave 
Etwnitsburg, MD 21727 



Michael D. Fay 
(301) 447-6671 



Department of Housing & Urban 

Development 
451 7th St. SW 
Washington, D.C. 20410 

Internal Revenue Service 
Training & Development Division 
2221 Jefferson Davis Hwy, Room 306 
Arlington, VA 22202 



Department of Interior, 
1 3th & C Streets 
Washington, DC 20240 



PPM 



Marybeth Budd 
(202) 755-5184 



Lori Gillespie 
(703) 557-2456 



Phyllis S. Jones 
(202) 343-8616 
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IKS. Employment Service 
601 D Street NW 
Washington, DC £j213 

Depatment of Labor 

Employment Standards Administration 

Div • of Personnel Development & 

Trai ni ng 
Washington, DC 20210 

Lister Hill Center for Biomedical 

Communication 
National Library of Medicine, NIH 
Bethseda, MB 20209 

National Institutes of Health 
Training Assistance Branch 
Bethesda, MB 21701 

Office of Personnel Management 
Instructional Systems and 

Technology 
1121 Vermont Ave NW 
Washington, DC 20044 



Walter J • Martin 
(202) 376-7M97 



Brendan Branley 
(202) 523-7036 



Craig Locatis 
J . P. Toothman 
(301) 496-6280 



Milt Tipperman 



Larry Westberg 
Patrice Walker 
Jack Weaver 
(202) 653*7055 



U.S. Senate Computer Center 
Educational Services & Support 

Section 
400 N, Capitol St. 
Washington, DC 20570 

Social Security Administration 
Office of Training 
6401 Security Boulevard 
Baltimore, MB 21235 

Veterans Administration 
810 Vermont Ave NW 
Washington, DC 20420 



Marian Saunders 
(202) 224-31^7 



Ray Halverson 
Rubin Shulik 
(301) 594-4526 
FTS 934-^526 

Leslie Schuman 
(202) 389-5796 



Other Organizations Represented in IGCBT 

U,S, Army Research Institute for the Behavioral and Social Sciences 

IKS. Army Training and Doctrine Command 

U.S. Army Logistics Management Center 

Centers for Disease Control 

U.S. Coast Guard 

U.S. Customs Service 

Defense Communication Agency 

Defense Languae Ins\itute Foreign Language Center 
Depatment of Energy 
General Accounting Office 
Gould S.E.L. 
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Government Printing Office 
HumflflO 

Institute for Defense Analysis 
Instructional Communications, Inc. 
Oak Ridge Associated Universities 
U.S. Postal Service 
Soil Conservation Service 



